The Lemon Fool

Why is the search facility so rubbish? I just searched Beerpig's Snug for Grand National, and received the response: `No suitable matches were found.` This is despite it being a hot topic of conversation every year since LF was founded.

Am I doing something wrong? If not, why doesn't it work?

Clitheroekid wrote:Why is the search facility so rubbish? I just searched Beerpig's Snug for Grand National, and received the response: `No suitable matches were found.` This is despite it being a hot topic of conversation every year since LF was founded.

Am I doing something wrong? If not, why doesn't it work?

How did you do that? 'Cos Beerpig's Snug isn't in the list of forums to search under advanced search!?! Not for me at least.

I get the same doing a board specific search search.php?keywords=Grand+National&fid%5B0%5D=29.

Not even the recent posts, entitled The Grand National.

Edit: Site:https://www.lemonfool.co.uk/ grand national externally via google finds more than none, such as from 2020 viewtopic.php?f=29&p=297234

Clitheroekid wrote:Why is the search facility so rubbish? I just searched Beerpig's Snug for Grand National, and received the response: `No suitable matches were found.` This is despite it being a hot topic of conversation every year since LF was founded.

Because it is a law of nature?

In my experience on site specific search facilities are always rubbish.

Try: 'Google http://www.lemonfool.co.uk: Grand National'

Instant BANG!

https://www.google.co.uk/search?client=opera&q=www.lemonfool.co.uk%3A+Grand+National&sourceid=opera&ie=UTF-8&oe=UTF-8

viewtopic.php?f=29&t=28235

XFool wrote:Try: 'Google http://www.lemonfool.co.uk: Grand National'

Instant BANG!

https://www.google.co.uk/search?client=opera&q=www.lemonfool.co.uk%3A+Grand+National&sourceid=opera&ie=UTF-8&oe=UTF-8

viewtopic.php?f=29&t=28235

Thanks, but having to go through that rigmarole does rather negate the purpose of the search facility.

Clitheroekid wrote:Thanks, but having to go through that rigmarole does rather negate the purpose of the search facility.

If you ever do find a "purpose" for the on site search facility, please let us know.

mc2fool wrote:

Clitheroekid wrote:Why is the search facility so rubbish? I just searched Beerpig's Snug for Grand National, and received the response: `No suitable matches were found.` This is despite it being a hot topic of conversation every year since LF was founded.

Am I doing something wrong? If not, why doesn't it work?

How did you do that? 'Cos Beerpig's Snug isn't in the list of forums to search under advanced search!?! Not for me at least.

The obvious way Clitheroekid could be doing that is the "Search this forum..." box just to the right of the "New Topic" button on viewforum.php?f=29.

But I suspect you're right about it being caused by Beerpig's Snug not being a searchable forum for advanced search - because the various non-advanced search facilities are probably just simplified interfaces to advanced search, so anything that isn't findable by advanced search probably isn't findable by any of the TLF search facilities.

But I agree that the TLF search facility is hopeless. If I want to find every post I've ever written on TLF, I can do an advanced search specifying Gengulphus as the author, and get essentially all of them: it currently gives 3737 results. But if I try to narrow the search down, e.g. by also specifying a word to search for, large numbers of results are missed. In particular, I almost always sign my posts with "Gengulphus", so one would expect an advanced search specifying Gengulphus as author and "Gengulphus" as a word they contain to produce almost as many results. But currently, such a search actually produces 1278 results - so it looks like getting on for 2/3rds of my posts aren't findable by words they contain. Currently, page 40 of the results of that search reveals the reason: everything I posted between 07 Feb 2017 17:22 and 23 Sep 2019 15:50 is missing from them. (Note that that page number can be expected to slowly increase as I post more - so as time goes by, that link can be expected to 'decay'.)

I've mentioned that problem before on this board (*), but no fix has appeared for it... I suspect the problem is to do with the size of the search indexes - certainly they became too unwieldy (and I strongly suspect too expensive to generate and/or store) at one point in the past. The solution appears to have been to cut down on the boards, date ranges, etc, that are indexed - and what isn't indexed isn't findable by search.

If I'm right and the limitations of TLF search are driven by the cost of the search indexes, this probably isn't a solvable problem. However, if that's the case, it would be good if the search results said something like "These search results might be be limited. More extensive results might be obtainable by a Google search, specifying 'site:lemonfool.co.uk' in addition to the words you're searching for."

(*) Edit: viewtopic.php?t=19727 is almost certainly the first such occasion.

Gengulphus

Clitheroekid wrote:Why is the search facility so rubbish?

It is an homage to The Motley Fool. Imitation is the sincerest form of flattery

Clariman wrote:

Clitheroekid wrote:Why is the search facility so rubbish?

It is an homage to The Motley Fool. Imitation is the sincerest form of flattery

I'm not so sure about that. Every forum I've ever been a member of is littered with complaints about how useless the site search facility is, and how using google works so much better.

Competent site admin usually explain that searching the forum database soaks up gazillions times more computing power than running the forum itself, so they have to severely limit the scope of an on-forum search facility, or start charging for membership to fund the computing power necessary. I dunno if this is actually true but i've heard/seen it said on several forums.

Mike4 wrote:I'm not so sure about that. Every forum I've ever been a member of is littered with complaints about how useless the site search facility is, and how using google works so much better.

Competent site admin usually explain that searching the forum database soaks up gazillions times more computing power than running the forum itself, so they have to severely limit the scope of an on-forum search facility, or start charging for membership to fund the computing power necessary. I dunno if this is actually true but i've heard/seen it said on several forums.

So how does searching with Google work, then? It's the same database on the same platform isn't it?

Adrian

AJC5001 wrote:

Mike4 wrote:I'm not so sure about that. Every forum I've ever been a member of is littered with complaints about how useless the site search facility is, and how using google works so much better.

Competent site admin usually explain that searching the forum database soaks up gazillions times more computing power than running the forum itself, so they have to severely limit the scope of an on-forum search facility, or start charging for membership to fund the computing power necessary. I dunno if this is actually true but i've heard/seen it said on several forums.

So how does searching with Google work, then? It's the same database on the same platform isn't it?

Adrian

I suspect the Google computer is a bit more powerful than the one Clariman and Stooze use to run TLF.

Just a wild guess, mind.

AJC5001 wrote:So how does searching with Google work, then? It's the same database on the same platform isn't it?

No, the TLF search uses an index on TLF built by the phpBB (the board software used by TLF) algorithm, whereas google uses ... guess what? ... indices on google built by google's algorithm, which is a lot smarter and a lot more comprehensive.

The snug is not under the search index so it won't find posts here.
However there is no reason for that, so I have turned it on. It may take a few days to catch up.

Regards

mc2fool wrote:

AJC5001 wrote:So how does searching with Google work, then? It's the same database on the same platform isn't it?

No, the TLF search uses an index on TLF built by the phpBB (the board software used by TLF) algorithm, whereas google uses ... guess what? ... indices on google built by google's algorithm, which is a lot smarter and a lot more comprehensive.

OK, so Google holds a separate index on it's own servers (presumably) built using it's own algorithms. In order to produce this index, Google will still need to access the TLF database. Surely this means that Google will be using TLF resources to access the database?

Could someone who understands these things tell me which bit of whose Terms & Conditions did I agree to that allows Google to access what I thought was data I was providing only to TLF?

If the Google search is so much better, then why not just abandon all attempts at maintaining a search facility within TLF (saving any processing needed to do so) and just provide a description of how to use Google search on the TLF site instead?

Adrian

AJC5001 wrote:

mc2fool wrote:

AJC5001 wrote:So how does searching with Google work, then? It's the same database on the same platform isn't it?

No, the TLF search uses an index on TLF built by the phpBB (the board software used by TLF) algorithm, whereas google uses ... guess what? ... indices on google built by google's algorithm, which is a lot smarter and a lot more comprehensive.

OK, so Google holds a separate index on it's own servers (presumably) built using it's own algorithms. In order to produce this index, Google will still need to access the TLF database. Surely this means that Google will be using TLF resources to access the database?

Google (indeed, all search engines) have what is called a Web Crawler that basically crawls around the web and indexes whatever it finds.

It does not access websites' own indices but rather indexes whatever it sees when reaching any public facing page; it's analogous to you visiting pages using your browser and noting the words used on those pages ... just a bit more comprehensive and a lot faster. It will use the same TLF resources as you do when you visit a page on TLF, no more and probably less, as it doesn't need to get all of the components of the page.

Of course, non-public facing pages, i.e. ones behind login requirements, can't be indexed by a web crawler, and there are ways of website administrators telling search engines not to access some/all public facing pages on their sites, should they want.

Could someone who understands these things tell me which bit of whose Terms & Conditions did I agree to that allows Google to access what I thought was data I was providing only to TLF?

If the Google search is so much better, then why not just abandon all attempts at maintaining a search facility within TLF (saving any processing needed to do so) and just provide a description of how to use Google search on the TLF site instead?

I'll leave Stooz/Clariman to answer those , but just to point out that TLF is a public-to-read website -- you only need to login to post -- so it's not just google that can read your posts, it's the whole world.

AJC5001 wrote: someone who understands these things tell me which bit of whose Terms & Conditions did I agree to that allows Google to access what I thought was data I was providing only to TLF?

Re access, see app.php/rules, specifically :

"When you post content (including but not limited to text, images, video or other media) on The Lemon Fool website (Site), you agree and represent that you have created that content or are making a limited quotation from others under "fair use" terms. Additionally when you post such content you expressly grant Lemonfool Ltd. a perpetual, irrevocable, fully transferable, non-exclusive, worldwide royalty-free licence to republish that content on our Site and to use/redistribute/make available that content in any format and on any platform, either now known or hereinafter invented anywhere in the world as part of an edited compilation or otherwise. As the licence is perpetual and irrevocable, this means that we do not have to remove your posts even if requested by you and even if you are no longer a registered user. By posting content, you warrant that you have the right to grant us the non-exclusive licence described above."

That permission (licence) which you have granted by using the TLF site is what allows TLF servers to display your content. It is still content that is owned by you, or is owned by the person you are quoting, but in granting that licence you are allowing TLF to display it to whoever TLF chooses to. At the moment TLF are displaying it to the whole world and I personally have no reason to think that might change, and that world includes Google's bots, and Microsoft's and GCHQ's and etc.

Regarding the issues about the organic search being pants, Stooz is looking into it once more. Don't hold your breath is my impression but he does have ideas.

regards, dspp

mc2fool wrote:...but just to point out that TLF is a public-to-read website -- you only need to login to post --

Apart from the Current Affairs & News boards (plus a few other places).

AJC5001 wrote:
Could someone who understands these things tell me which bit of whose Terms & Conditions did I agree to that allows Google to access what I thought was data I was providing only to TLF?

If it concerns you then perhaps you should read, and have ongoing access to, Terms and Conditions you agree to. You wouldn't then have to worry about what you "thought" you had agreed to, but instead "know" what you had agreed to.

XFool wrote:

mc2fool wrote:...but just to point out that TLF is a public-to-read website -- you only need to login to post --

Apart from the Current Affairs & News boards (plus a few other places).

Okay so for Current Affairs & News mc2fool should perhaps have written:

You must be a registered member and logged in to view the replies in this topic.

Although he later wrote Of course, non-public facing pages, i.e. ones behind login requirements which you may have missed.

As it happens, one appears to be able to see the OPs there come what may.

Do you know where your a few other places might be (other than the Mod Room)?

PinkDalek wrote:

XFool wrote:

mc2fool wrote:...but just to point out that TLF is a public-to-read website -- you only need to login to post --

Apart from the Current Affairs & News boards (plus a few other places).

Okay so for Current Affairs & News mc2fool should perhaps have written:

You must be a registered member and logged in to view the replies in this topic.

Although he later wrote Of course, non-public facing pages, i.e. ones behind login requirements which you may have missed.

As it happens, one appears to be able to see the OPs there come what may.

As I pointed out elsewhere, the LF posted description on the board says posters' should bear in mind that it is publicly visible. It isn't. Unless by "publicly visible" they mean it can be seen if you are logged on to your LF account, but don't need additional registration as with PD.

PinkDalek wrote:Do you know where your a few other places might be (other than the Mod Room)?

Ignoring obviously private areas (account settings etc.) there is the Brokers and Share Dealing board. I believe the reason for this has been given in the past.

But then, this "not publicly visible" seems incompatible with "not publicly visible" wrt Current Affairs & News. It's confusing, IMO.