Search suggestions and minimum search results

I'm wondering if there is a setting that specifies the minimum search results that it can find before the "did you mean" function is called. I had a quick look through the code and could not see anything defined for this - mind you, I wasn't looking really hard. 24.67.76.75 21:32, 6 February 2013 (UTC)

We are simply plugging into the existing MW suggestions framework there, so it relies on whenever MW decides to call hasSuggestion() method. I cant remember any more details right now. Svemir Brkic (talk) 14:22, 7 February 2013 (UTC)

Unknown field error in indexer

Background: I have Sphinx completely setup and working. In the process of using Sphinx I realize I have a column holding text that I would like to filter against. Potential text values in this field are stored in a second table called counties. The counties table does not have a int ID column, so I added it and set it as the primary key. Using a left join I attached it to the primary table and added the field to the field list using 'as CountyID' statement. To verify the syntax, I ran the query in MYSQL workbench,everything worked.

Problem: I moved the query over to Sphinx Config file and the indexer will not recognize the new column. If I replace that field with any other field in the counties table I do not receive an error. It is specific to the added column. I even went so far as renaming the column. That did not work.

Any suggestions? 96.35.240.46 18:41, 22 February 2013 (UTC)

Sphinx is run as services ? If true, perhaps tries to restart it. Lutch 2904 (talk) 14:04, 3 April 2013 (UTC)

Search suggestions do not include other namespaces

Hello,

We are trying to get the search suggestions box that appears as you type in the search box to include suggestions from non-default namespaces. The regular search includes all namespaces, but for some reason the suggestion box does not. I've tried a few things but nothing seems to pertain specifically to the suggestions returned rather than the search results as a whole.

For example, we have a namespace called "Tech", and in it an article named "Printers". If we just type "Print" into the suggestions box the article is not suggested, but if we type "Tech:Print" it is suggested. Essentially I'd like the suggestions to automatically assume that "Tech:" is supplied, as most of our content is in that namespace.

MediaWiki version: 1.20.2

Sphinx version: 2.0.6

SphinxSearch version: 0.8.5 Tmitera (talk) 18:56, 4 April 2013 (UTC)

With assistance from a kind soul on the MediaWiki Users forums I've got the answer.

In PrefixSearch.php you must change the following block of code:

// Find a Title which is not an interwiki and is in NS_MAIN
        $title = Title::newFromText( $search );
        if( $title && $title->getInterwiki() == '' ) {
            $ns = array($title->getNamespace());
            if( $ns[0] == NS_MAIN ) {
                $ns = $namespaces; // no explicit prefix, use default namespaces
            }
            return self::searchBackend(
                $ns, $title->getText(), $limit );
        }

...to the following:

// Find a Title which is not an interwiki and is in NS_MAIN
        $title = Title::newFromText( $search );
        if( $title && $title->getInterwiki() == '' ) {
            $ns = array($title->getNamespace());
            if( $ns[0] == NS_MAIN ) {
                $ns = $namespaces; // no explicit prefix, use default namespaces
            }
            global $wgNamespacesToBeSearchedDefault;
            $results = array();
            foreach($wgNamespacesToBeSearchedDefault as $k => $v)
            {
                $results = array_merge($results, self::searchBackend(array($k), $title->getText(), $limit ) );
            }
            return $results;
        }

Credit: Didster (http://www.mwusers.com/forums/showthread.php?22328-Can-search-suggestions-contain-results-from-non-default-namespaces&p=68347&viewfull=1#post68347) Tmitera (talk) 21:43, 12 April 2013 (UTC)

Site broke down after enabling sphinxsearch

Dear,

Hello, i've just installed SphinxSearch, however after i put in the extension,
$wgSearchType = 'SphinxMWSearch';
require_once( "$IP/extensions/SphinxSearch/SphinxSearch.php" );

my site broke down, with blank screen, at the moment, i think it's because not compatible with this version of MW?

cause after i comment out that require sphinxsearch.php line, everything went back to normal.

Thanks in advance for any help.
MediaWiki 1.20.4

Sphinx 2.0.6

SphinxSearch 0.8.5(got it through SVN link of the extension) Nick1092 (talk) 07:51, 23 April 2013 (UTC)

Same here with git branch REL1_22. Unikum (talk) 10:25, 2 November 2013 (UTC)

Blank screen usually means there is some error. You should be able to see in your php error log. If you post the error message here, maybe someone will be able to help you. Svemir Brkic (talk) 01:37, 27 January 2014 (UTC)

See Extension:SphinxSearch#Step_8.1_-_Extension_Preparation_-_Sphinx_PHP_API

Solved after this (Ubuntu 12.04.3):

# ln -s /usr/share/sphinxsearch/api/sphinxapi.php /var/www/wiki/extensions/SphinxSearch/sphinxapi.php

Unikum (talk) 03:45, 27 January 2014 (UTC)

HTTP 301 - Moved permanently

search is sometimes case sensitive

My site it thepetwiki.com If I search for 'adopting a feral dog' it doesn't bring back the article page but 'Adopting a Feral Dog' does. However both 'our natural pets' and 'Our Natural Pets' brings back results. Why would this be so? The sphinx is running. Is the default Mediawiki search engine conflicting with it? Any ideas? Thanks Tech (talk) 09:15, 9 June 2013 (UTC)

I just tried on your site and both "adopting a feral dog" and "Adopting a Feral Dog" bring back the same search results. The only difference in when you press enter instead of clicking Fetch. In that case the lowercased search still brings back the usual results, but the correctly cased search will redirect you to the actual article, because that is an exact match. Svemir Brkic (talk) 12:46, 9 June 2013 (UTC)

When I try 'adopting a feral dog' clicking 'fetch' and 'enter' both give me search results and not the article page. Doesn't this mean it is case sensitive?

I'm not sure what you mean 'The only difference in when you press enter instead of clicking Fetch.' Tech (talk) 07:31, 24 June 2013 (UTC)

I forget how the direct title lookup used to work, but it is possible it is case sensitive. You seem to be using a relatively old version of the extension. If possible, test it with the lastest version, and make sure to set $wgSearchType = 'SphinxMWSearch'; Svemir Brkic (talk) 11:16, 24 June 2013 (UTC)

If you haven't enabled auto-complete using SphinxSearch ($wgEnableSphinxPrefixSearch, never tried it myself, though), then MediaWiki's default is case-sensitive, afaik. I believe Wikipedia overcomes this by using Extension:TitleKey, which is enabled there. FreedomFighterSparrow (talk) 03:43, 26 July 2013 (UTC)

Adding wiki prefix in Installation Step 2

I'm currently installing sphinx and the extension for my website. I've gone through most of it but I cannot figure out where to add my database prefix for database. In step 2 of the installation it says this "Update table names in SQL queries if your MediaWiki installation uses a prefix (backslash line breaks may need to be removed if the indexer step below fails)".

The following is the default setting in the sphinx.conf file. Where exactly do I add my prefix "Wiki_". As of now trying to run the indexer, I get a permission denied error which Im thinking could be b/c of this problem.

pre-query, executed before the main fetch query

sql_query_pre = SET NAMES utf8

main document fetch query - change the table names if you are using a prefix

sql_query = SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM page, revision, text WHERE rev_id=page_latest AND old_id=rev_text_id Wikimanz (talk) 10:45, 12 July 2013 (UTC)

You would need to change the sql_query line like this:

sql_query = SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM Wiki_page, Wiki_revision, Wiki_text WHERE rev_id=page_latest AND old_id=rev_text_id Svemir Brkic (talk) 00:52, 30 August 2013 (UTC)

Run script to restart the daemon process automatically on server restart.

'...it is critical to make sure the daemon process is started every time the server is restarted.' Every time my server is restarted I have to remember to manually run the script to re-start the daemon process. Is there a way to set this up so it is automatic? My site is on c-panel. Thanks LTech (talk) 08:48, 2 September 2013 (UTC)

if your distro is systemd related review the service file and make sure it has ExecStart/Stop and Restart fields defined. For sysv/init.d types, you can consider installing monit. Skunark (talk) 02:55, 30 December 2013 (UTC)

Mediawiki Search Text Within code blocks

Hi folks, I just installed and setup SphinxSearch Extension just so that i could Search Text Within code blocks: http://stackoverflow.com/questions/13262368/mediawiki-search-text-within-code-blocks, but i still can't. Can anyone suggest why?

SphinxSearch is working (because i am not getting any errors while searching on my mediawiki)

My versions:
Product	Version
MediaWiki	1.19.2
PHP	5.4.4 (apache2handler)
MySQL	5.5.25a
SphinxSearch (Version 0.8.5)
Sphinx Engine: sphinx-2.0.5-release-win64-id64-full

Abarik (talk) 03:06, 19 September 2013 (UTC) Hello Abrik. Are you sure SphinxSearch is working? No errors does not mean it is working. Even then, when I use our test wiki with MW-1.21.1 with only SyntaxHighLight enabled and create a page called Test and save the page with:

<source>
ifconfig
</source>

The default search engine will find ifconfig. Also with SphinxSearch a hit is produced. Trye to disable all extensions on your wiki but not SyntaxHighLight and see if it works then. You can also try a clean install of MW-1.21.1 so you can do some testing, maybe you need to upgrade to MW-1.21.1. Jongfeli (talk) 07:55, 19 September 2013 (UTC)

Thanks Jongfeli. I might have to upgrade my MW to 1.21.1. Thanks. Do you know how to verify if SphinxSearch is actually working from MW? Abarik (talk) 22:05, 25 September 2013 (UTC)

When you create a page with the word "birds" on it and then run the indexer again. Then search for "bird". With SphinxSearch you should get a hit on "birds". The default search engine will not produce a hit in this case. If you are using "Search suggestions" with Aspell try a search on "proccesor". You should get a "Did you mean: processor". If you don't get a "Did you mean:" your Search suggestions are not working. Jongfeli (talk) 06:09, 26 September 2013 (UTC)

Another way to confirm if your wiki is using the default MW search vs SphinxSearch is to look at the search results:

• Search result groups:
  • MW - results will be broken up into two sections: 'Page title matches' and 'Page text matches'.
  • SS - results are not broken into groups.
• # of results returned info in gray bar:
  • MW - Says something like 'Showing below up to 20 results starting with #1'
  • SS - Says something like 'Results 1 - 20 of 788 for xxx'
• Actual results - Comment out the SS info in LocalSettings.php. Do the search results change as you switch back and forth? Although it might be hard to tell if you don't have much content. It is blatantly obvious with the amount of content we have. Mattsmith321 (talk) 03:43, 27 September 2013 (UTC)

Sphinx not reallocating custom namespaces

Hello,

I inherited maintaining an internal wiki, and one of the things I encountered was that a some people use the wiki to keep logs of their day's activities. This is all well and good, until we ran into the problem of these logs needlessly clogging up the search with 200+ logs (and the added benefit of their title beginning with a number so they _always_ show up first in a search query).

My solution is to put all of these existing logs into a new namespace so that they can be searched separately from the main namespace. I was able to create the custom namespace, move the existing pages through mysql queries, and reindex the search database, but the new namespace still showed up in the mainspace search, even though using the Extension talk magic word clearly showed these logs were in the new namespace.

What I discovered is that Sphinx is still indexing these pages into the main namespace, and when I disable the extension the new namespace searches correctly.

I still want to use the extension, so does anyone have any ideas how I can make sphinx reindex existing pages into a new custom namespace?

Thanks, Devin Silversonic (talk) 15:16, 18 October 2013 (UTC)

No worries I figured it out: /var/data/sphinx/* needed to be scrubbed and reindexed. 198.45.120.122 17:19, 21 October 2013 (UTC)

sphinx.conf for postgresql

# data source definition for the main index
source src_wiki_main
{
	# data source
	type		= mysql
	sql_host	= localhost
	sql_db		= #replace with your db name
	sql_user	= #replace with your db username
	sql_pass	= #replace with your db password
	# these two are optional
	#sql_port	= 3306
	#sql_sock	= /var/lib/mysql/mysql.sock

	# pre-query, executed before the main fetch query
        # Postgresql: setting the encoding is probably unnecessary, while
        # setting the schema search path spares us qualifying our tables
	sql_query_pre	= SET client_encoding="UTF8";  SET search_path TO mediawiki,public;
	
	# main document fetch query - change the table names if you are using a prefix
	 # Postgresql: text == pagecontent on postgresql and oracle
	sql_query	= SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM page, revision, pagecontent WHERE rev_id=page_latest AND old_id=rev_text_id


	# attribute columns
	sql_attr_uint	= page_namespace
	sql_attr_uint	= page_is_redirect
	sql_attr_uint	= old_id

	# collect all category ids for category filtering
        # Postgresql: can't set the search path first, so we have to 
        # be explicit
	sql_attr_multi  = uint category from query;  SELECT cl_from, page_id AS category FROM mediawiki.categorylinks, mediawiki.page WHERE page_title=cl_to AND page_namespace=14

	# used by command-line search utility to display document information
	sql_query_info	= SELECT page_title, page_namespace FROM page WHERE page_id=$id
}

# data source definition for the incremental index
source src_wiki_incremental : src_wiki_main
{
	# adjust this query based on the time you run the full index
	# in this case, full index runs at 7 AM UTC
        # Postgresql: changed the date formatting
        sql_query	= SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM page, revision, pagecontent WHERE rev_id=page_latest AND old_id=rev_text_id AND page_touched>=current_date + cast('07:00:00+00' as time with time zone)

	# all other parameters are copied from the parent source
}

Some pages arent found - and the "second" search is different than the first one

Hey guys,

ive 2 questions.

First of all, i installed sphinx, everthing worked fine - so much better than standard wiki search engine :D.

But:

Today i recognized, that the "first search" is different as "second" (Picture 1 and 2).

http://s1.directupload.net/images/131104/ajqaj4z6.jpg http://s7.directupload.net/images/131104/u63dfpvl.jpg

I have tested it - in the "first" search sphinx is uses too, but delivers different (less?) results. WHy :D?

And my second issue is:

One site (maybe more, not tested) dont get found.

Here a two Pictures:

http://s7.directupload.net/images/131104/ecroc3gc.jpg http://s14.directupload.net/images/131104/u32gzgpf.jpg

i also tried it without namespaces - didnt work.

But when i ran the search outta cmd:

F:\sphinx\bin>search.exe --config F:\sphinx\sphinx.conf "Maschinenbeschreibung"

Everything is alright - Site is found.

Would be very appreciated if someone could help me out here.

Regards

MediaWiki 1.17.0 PHP 5.3.3 (apache2handler) MySQL 5.1.49-community

SphinxSearch (Version 0.8.5) Babacuekk (talk) 14:37, 4 November 2013 (UTC)

sry i forgot:

I have enabled "automatic Wildcard".

• Maschinenübersicht* delivers no replies too (in wiki - in cmd it does) 194.156.246.226 14:55, 4 November 2013 (UTC)

For issue #1 where the search results are different: From your pictures, it appears that you are confirming that the "Search" button was clicked in both examples (not "Go" or the enter key), correct? If so, what do the URLs look like for both search result examples? It seems like I've seen some difference depending on where you are coming from. If they are different, it would indicate that there is something in the code behaving differently. Not sure if that got addressed in a later version. Mattsmith321 (talk) 19:53, 5 November 2013 (UTC)

For both issues, I would recommend performing your Sphinx testing with SphinxQL. SphinxQL can be tweaked to match the MediaWiki search results, whereas search.exe is a little less flexible. You may also need some additional parameters in the query to fully get it to match the MediaWiki configuration. Using SphinxQL, you can take away the MediaWiki layer and get a better idea of exactly how Sphinx is working behind the scenes (although is probably less needed since you don't have many results returned.). Mattsmith321 (talk) 20:00, 5 November 2013 (UTC)

I have the same issue. The quick search box (Vector skin) returns sphinx results but the Special:Search page does not.

In my example, I search my wiki for "server".

The quick search box hits sphinx and returns all pages containing the word "server". The URL looks like this:

/index.php?search=server&title=Special%3ASearch

The search box on the Special:Search page finds no results, as though it's doing a "Go" search for page names and not hitting sphinx. The URL looks like this:

/index.php?title=Special%3ASearch&profile=default&search=server&fulltext=Search

Removing "&profile=default" from the second URL results in a good sphinx search.

Does anyone know where the profile querystring is coming from, and how to disable it for the Special:Search page? Lsilverman (talk) 21:01, 30 December 2013 (UTC)

Let me add that the intent of what Special:Search is supposed to do is unclear to me when a third-party search engine like Sphinx is in use. It's possible that the authors of MediaWiki intended Special:Search to behave as it always did regardless of third party search engines, third party search engines are only supposed to work with the quick search box. Or, perhaps there is something the author of a search extension is supposed to do to change the default behavior of the Special:Search page, and that work was not done for the Sphinx MediaWiki extension.

I would have expected that the two search boxes should work the same, and the user would have to intentionally choose alternate settings to get back to the "default" style of searching the wiki. At a minimum, I would expect that it should be possible to conduct a full text search using sphinx from Special:Search, but I cannot figure that out. My wiki users also assume this, as they keep asking me why the searching on Special:Search doesn't find any pages like they are used to finding with the quick search box.

I would very much like to make Special:Search behave, by default, exactly as the quick search box behaves. Lsilverman (talk) 21:16, 30 December 2013 (UTC)

Still hoping for a response to this. Why does the presence of the "profile" querystring on Special:Search cause a bypass of the Sphinx search engine?

I found that if I modify includes/specials/SpecialSearch.php around line 1092 and comment out the line:

// $out .= Html::hidden( 'profile', $this->profile ) . "\n";

...then I get the desired behavior.

I don't like changing the source code in this manner, but it's the only solution I've found thus far. Lsilverman (talk) 21:16, 13 January 2014 (UTC)

Babacuekk: 1.17 is a rather old version of MW. Any chance you could update? This could be an issue with how the search box is generated in the skin, or maybe how encoding is done - it would help if you would post url arguments for the two cases.

Lsilverman: What is your version of MW? On 1.22, even though I do get the profile=default in the URL too, the results are the same in both cases. If you check your sphinx.conf, you can see where the query.log is. Watch that file to compare the queries in two different cases.

As for the behavior of Special:Search, the behavior you expect is the correct one, and that is how it works for me. Maybe there is some setting interfering with things, or some of the code is different in your version of MW. Svemir Brkic (talk) 01:23, 27 January 2014 (UTC)

Babacuekk: I'm running 1.22.

When on the Special:Search page with "Content pages" link selected, searches do not hit the sphinx index; nothing in the query_log. Searches get routed to the default search, which is only querying page names.

The URL of the "Content pages" link is: /w/index.php?title=Special:Search&search=&fulltext=Search&profile=default&redirs=1

While we're on this related topic, it also seems to be the case that the quick search text box atop the Vector skin runs the default Mediawiki search when the Enter/Return button is used to conduct the search. Lsilverman (talk) 17:21, 27 January 2014 (UTC)

Do you have "$wgSearchType = 'SphinxMWSearch';" BEFORE require_once "$IP/extensions/SphinxSearch/SphinxSearch.php"; in your LocalSettings? That is the only var. that needs to be enabled before the rest of the code is included, though I am considering changing that in the next version.

As for the quick search, try setting $wgEnableSphinxPrefixSearch = true; after you include the extension in LocalSettings. Svemir Brkic (talk) 17:27, 27 January 2014 (UTC)

Yes. Here's the relevant section:

# Sphinx Search
$wgSearchType = 'SphinxMWSearch';
require_once("$IP/extensions/SphinxSearch/SphinxSearch.php");
$wgSphinxSearch_index = "dev_wiki_index";
$wgSphinxSearch_index_list = "dev_wiki_index";
$wgEnableMWSuggest = true;
$wgEnableSphinxPrefixSearch = true;
$wgSphinxSuggestMode = 'enchant';

Lsilverman (talk) 17:32, 27 January 2014 (UTC)

Hi,

I just migrated to MW 1.22 and also got forced to migrate the SphinxSearch extension to 0.8.5 + Sphinx 2.0.4 as recommended. Since then, any search uses the default MW as stated in this thread. I do have the $wgSearchType set before the inclusion of the SphinxSearch extension. Looking at the log is not help since it doesn't even reach the SphinxSearch page code at all.

Any news on this issue ? Egemme (talk) 03:20, 3 May 2014 (UTC)

Hi,

I just migrated to MW 1.22 and also got forced to migrate the SphinxSearch extension to 0.8.5 + Sphinx 2.0.4 as recommended. Since then, any search uses the default MW as stated in this thread. I do have the $wgSearchType set before the inclusion of the SphinxSearch extension. Looking at the log is not help since it doesn't even reach the SphinxSearch page code at all.

Any news on this issue ? Egemme (talk) 03:20, 3 May 2014 (UTC)

No updates on this? Same problem... 64.148.137.131 23:43, 14 July 2014 (UTC)

Categories in search results

Hi,

A great feature would be if you where able to see which category or categories the page in the search results belonged to. This is something I think is lacking for the regular search in mediawiki as well.

Is this something that is possible to add?

BR, Daniel 195.60.68.155 11:29, 14 November 2013 (UTC)

No Search Results

CentOS 6.5 64-bit PHP Version 5.3.3 MySQL 5.1.71 MediaWiki 1.22.0 Sphinx 2.0.4 SphinxSearch 0.8.5

This is a clean install. I did import my database from an older version of Mediawiki (1.15), but the upgrade script updated it without any problems.

I can run the indexer with success. I can search the index from the command line. The searchd is definitely listening on port 9312. When I run a search from the command line, I get almost 200 matches for a term, but when I try to run the same search from the MediaWiki search form, I get nothing. 67.78.218.162 23:10, 27 December 2013 (UTC)

No Search Results - HELP!

CentOS 6.5 64-bit
PHP Version 5.3.3
MySQL 5.1.71
MediaWiki 1.22.0 (/var/www/html/wiki)
Sphinx 2.0.4 (/usr/local/sphinx)
SphinxSearch 0.8.5 (/var/www/html/wiki/extensions/SphinxSearch)

This is a clean install. I did import my database from an older version of Mediawiki (1.15), but the upgrade script updated it without any problems.

I can run the indexer with success. I can search the index from the command line. The searchd is definitely listening on port 9312. When I run a search from the command line, I get almost 200 matches for a term, but when I try to run the same search from the MediaWiki search form, I get nothing. Jeremyflint (talk) 23:13, 27 December 2013 (UTC)

I was able to run test.php from the extensions folder, using the Sphinxapi.php file that I copied to the extension folder. So I know that the sphinxapi is seeing searchd properly. Seems to be something keeping mediawiki from handing off the search term to searchd. Jeremyflint (talk) 20:13, 28 December 2013 (UTC)

Please check your sphinx.conf for a line like this:

query_log = /var/log/sphinx/query.log

Watch that file to see if the query is getting to sphinx. Svemir Brkic (talk) 01:28, 27 January 2014 (UTC)