Typo3 MnogoSearch Engine Setup for Ubuntu 09.10 and 10.04
Base for the setup is the manual.swx file from Dmitry and http://software.rde.de/mnogo-die-erste.html . I will just describe the adoptions for Ubuntu 09.10 and 10.04.
Installation Requirements
The following packages are required for compiling mnogosearch.
automake - installed autoconf - installed gcc - installed ispell - to install ingerman - to install libmysqlclient16-dev - to install mc - for editing, installed mysql - installed php5-dev - to install zlib - installed
Now the source code is copied to /opt and unpacked.
$ sudo cp mnogosearch-3.3.9.tar.gz /opt/mnogosearch-3.3.9.tar.gz $ cd /opt $ sudo tar xzf mnogosearch-3.3.9.tar.gz $ cd mnogosearch-3.3.9
Next step is to modify, compile and install mnogosearch.
Edit files to exclude PGSQL (manual.swx, page 13) Edit file "configure.in" Line 1017, AC_DEFINE([HAVE_PGSQL], [1], [Define if you want to use PostgreSQL]) change "1" to "0" Edit file "configure" to exclude PGSQL Line 26547, #define HAVE_PGSQL 1 # 1 -> 0 $ ./configure ... # manual page 13 Now open include/udm_autoconf.h and locate /* #undef HAVE_PGSQL */ Line 133, change to "#undef HAVE_PGSQL" $ make $ sudo make install
Next step is to modify, compile and install the PHP extension. In order to check the proper installation look in the TYPO3 Install tool under 8: phpinfo(). Under chapter Configuration it should show the block mnogosearch with Support enabled.
cd /opt/mnogosearch-3.3.6/php Edit php_mnogo.c - AFTER line 30 #include "php.h" Insert #undef HAVE_PGSQL $ phpize Configuring for: PHP Api Version: 20041225 Zend Module Api No: 20060613 Zend Extension Api No: 220060519 $ sudo ./configure --with-mnogosearch=/opt/mnogosearch $ sudo make $ sudo make test $ sudo sudo make install Installing shared extensions: /usr/lib/php5/20060613+lfs/ OK Just for special case: Find PHP5 extension dir: $ php -i | grep extension_dir $ sudo cp modules/mnogosearch.so /usr/lib/php5/20060613+lfs/mnogosearch.so
Edit php.ini for Command Line Interpreter testing.
/etc/php5/apache2/php.ini and /etc/php5/cli/php.ini Line 503 extension_dir = "/usr/lib/php5/20060613+lfs" At the end of "Dynamic Extensions" add: Line 616 extension=mnogosearch.so $ sudo /etc/init.d/apache2 restart $ php -m # show all loaded modules First create the user '''_cli_mnogosearch''' in Typo3 backend. $ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch help # for test
Create database index.
$ mysql -u root -p rudisflugis Enter password: **** > create database mnogosearch; > grant all privileges on mnogosearch.* to 'root'@'localhost' identified by 'rudiscomplexpassword'; > exit; $ find /usr -name "cli_*" /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -w -n &>/dev/null or $ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -w -v 3 # verbose or clear index $ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -x -Cw
TYPO3 setup
Now login the TYPO3 backend and create a new user _cli_mnogosearch. The password must be set, but is not used.
Admin-Tools/Administration Result usernames: - click on Add + symbol Enter usename "_cli_mnogosearch" Enter password "password" Save record
In order to allow global extensions, it must be enabled once with the Install-Tool.
Start Install-Tool Select 5:All Configuration Set a mark at [allowGlobalInstall] At the bottom of the windows click on button "Write to localconf.php"
Next start the extension manager.
Mode: Import extensions Download first from www.typo3.org the extensions as *.t3x file: page path, ver. 0.1.6 Dmitry Dulepov universal page browser, ver. 1.1.0 Dmitry Dulepov mnogosearch, ver. 2.2.2 Dmitry Dulepov Select function "Import extensions" in field "UPLOAD EXTENSION FILE DIRECTLY (.T3X)" select pagepath_0.1.6.t3x ... to location "Global (typo3/ext/)" mark "Overwrite any existing extension!" click on "Upload extension file" No file uploaded! Probably the file was too large for PHPs internal limit for uploadable files. ------------- unfortunately it does not work, also with ver. 4.2.12 ----------- Solution: $ sudo mkdir /var/run/typo3-dummy $ sudo chown www-data:www-data /var/run/typo3-dummy Workaround: First install "local" from repository, than move to global folder. Type in the "Look up" field: universal page browser: Look up for "page" mark "Overwrite any existing extension!" click on "Upload extension file" Repeat for Look up "pagepath" Repeat for Look up "mnogosearch" Then move the 3 folders from /var/lib/typo3conf/ext to /usr/share/typo3/typo3_src-4.2.12/typo3/ext $ sudo chown -R www-data:www-data /usr/share/typo3/typo3_src-4.2.12/ext $ cd /var/lib/typo3-dummy $ sudo chmod -R a+w fileadmin typo3conf typo3temp uploads
When the last extensions mnogosearch was installed, the extension manager tells Installing mnoGoSearch: DATABASE NEEDS TO BE UPDATED and Before the extension can be installed the database needs to be updated with new tables or fields. Please select which operations to perform:
Add tables mark at CREATE TABLE tx_mnogosearch_indexconfig mark at CREATE TABLE tx_mnogosearch_urllog Create folder mark at Create directory "typo3temp/mnogosearch/var" Click on "Make updates" You will see a new windows with: Active status: The extension is installed (loaded and running)! then check: Path to mnoGoSearch installation: /opt/mnogosearch Database connection string: mysql://user:pass@host/dbname/?dbmode=multi mysql://root:o****@localhost/mnogosearch/?dbmode=multi # ReadMe_Typo3.txt Charset for database: utf-8 Charset for results: utf-8 mark at Hide internal tables in List module then click on "Update" then check: Installation status: Type of install: Global - Global extensions (typo3/ext/) - Available for shared source on server (Dynamic). On the command line test the function: NOTE: options -h, --help and -? does NOT work, use instead just "help"! rudi@rudi2u:~$ /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch help Usage: cli_mnogosearch.phpsh [-n] [-w] This script reindexes TYPO3 web sites as defined by current TYPO3 configuration. It accepts the following options: -c Only check and create database if necessary. Do not reindex. -d Display generated indexer configuration and exit. -n Force reindexing of new URLs (normally should be set) -p pid Process indexing configuration only from this pid -w Create statistic for misspelled words. Useful only if . Ispell dictionaries are included to mnoGoSearch . configuration (see mnoGoSearch documentation) --dry-run Show what will be done (not applicable to -d and -E) -h, --help, -? Display this help message -x Pass the argument to mnoGoSearch indexer -v level Be verbose. Level is 0-5. Default is 0 (complete silence)
Update to TYPO3 4.2.12
Because the included (Ubuntu 0910) TYPO3 version was 4.2.6 it should be updated:
Download from www.typo3.org file typo3_src-4.2.12.tar.gz Unpack with archive manager in folder /usr/share/typo3 Copy global extensions from old old installation: $ sudo cp -R /usr/share/typo3/typo3_src-4.2/ext/* /usr/share/typo3/typo3_src-4.2.12/ext $ sudo chown -R www-data:www-data /usr/share/typo3/typo3_src-4.2.12/ext $ cd /var/lib/typo3-dummy/ $ sudo rm typo3_src $ sudo ln -s /usr/share/typo3/typo3_src-4.2.12 typo3_src $ cd /usr/share/typo3/typo3_src-4.2.12 $ sudo chown -R root:www-data * $ sudo chmod -R g+w * Calling the Install-Tool from Backend gives several warnings, but it works.
TYPO3, mNogoSearch setup
In the TYPO3 backend click on Web/List and add a new page named Konfigurations-Tools.
Typo3 4.2.x Click on globe and "new", new page with wizzard, page title is "Konfigurations-Tools" - save and exit Click on right side on "Create new record", select type "mnoGoSearch indexing configuration" and insert it below the "Master_Template". The title will be "mnoGoSearch-IndexConfig" Indexing path: http://www.rudiswiki.de/flugis/ # rudisflugis, IMPORTANT: trailing "/" otherwise other web pages from this domain (e.g. wiki1) will be indexed. Typo3 4.3.x Select Web/List mode. Click on the domain root in the tree. In the right window click on the "New record" icon. Select "new page" and name the title "Konfigurations-Tools" Click in the page tree on "Konfigurations-Tools", then in the right window on the icon "Create new record". Next click on "mnoGoSearch indexing configuration" Configuration type: server Indexing path: http://www.rudiswiki.de/flugis Title: mnoGoSearch root indexer Index method: Allow Index period: (use cron job) Save the record.
In case you changed the indexing path, please lookup with phpmyadmin if you have just the desired inding pathes in the table xxx. If there are more than that, delete the unwanted indexing pathes.
First Index run:
clear index $ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -x -Cw or index $ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -v 3 -w indexer from mnogosearch-3.3.9-mysql-DB2-solid-SAPDB-ibase-ctlib-freetds-oracle8-oracle8i started with '/tmp/mnogosearch-olrMiC' [8230]{01} URL: http://localhost/typo3 [8230]{01} ROBOTS: http://localhost/robots.txt [8230]{01} URL: http://localhost/typo3/ [8230]{01} URL: http://localhost/typo3/link-seite.html ... [8230]{01} URL: http://localhost/typo3/fileadmin/download/impressum0.html [8230]{01} Writing words (177107 words, 3978987 bytes, final). [8230]{01} The words are written successfully. (final) [8230]{01} Done (358 seconds, 562 documents, 8242536 bytes, 22.48 Kbytes/sec.) Calculating word statistics Word statistics done
Add Search Mask to web
In order to enter search words, it needs a search mask. This is imported from a master web, schnelle_suche.t3d CSS
Exclude anything else content from index
In order to exclude anything else content from the index, use the mark up <NOINDEX> .. </NOINDEX> in the proper template xxx.html file, e.g. fileadmin/templates/rudi-folgeseite-index.html.
mNogoSearch PageBrowser
In order to use the page browser to navigate in more than one page of search results, it has to be setup first.
Edit file /var/lib/typo3-dummy/typo3/ext/mnogosearch/static/mnoGoSearch/constants.txt Add on top of the file from /var/lib/typo3-dummy/typo3_src/typo3/ext/pagebrowse/constants.txt: plugin.tx_pagebrowse_pi1 { # cat=plugin.tx_pagebrowse_pi1; type=string; label=Template file templateFile = EXT:pagebrowse/res/template.html } Edit file /var/lib/typo3-dummy/typo3/ext/mnogosearch/static/mnoGoSearch/setup.txt Add on top of the file from /var/lib/typo3-dummy/typo3_src/typo3/ext/pagebrowse/setup.txt: plugin.tx_pagebrowse_pi1 = USER plugin.tx_pagebrowse_pi1 { # USER_INT support includeLibs = EXT:pagebrowse/pi1/class.tx_pagebrowse_pi1.php userFunc = tx_pagebrowse_pi1->main # Template file templateFile = {$plugin.tx_pagebrowse_pi1.templateFile} # Extra parameters to the query string. Must start with & if not empty extraQueryString = # Number of page links to show before the current page pagesBefore = 3 # Number of page links to show before the current page pagesAfter = 3 # GET variable name for page pointer. Examples: "page" or "tx_exykey_pi1|page". Notice that array separator is pipe (one level only!) pageParamName = tx_pagebrowse_pi1|page # Enables section for "more" pages. This section is shown after links to next pages, normally like three dots (1 2 3 ...). Notice that you can also hide it by emptying corresponding template section. enableMorePages = 1 # Enables section for "less" pages. This section is shown after links to next pages, normally like three dots (1 2 3 ...) Notice that you can also hide it by emptying corresponding template section. enableLessPages = 1 } Edit /var/lib/typo3-dummy/typo3/ext/mnogosearch/resources/styles.css Add on top of the file from /var/lib/typo3-dummy/typo3_src/typo3/ext/pagebrowse/styles.css: /* der Dmittry pagebrowser */ .tx-pagebrowse { text-align: center; list-style: none inside none; padding: 0 0; margin: 0px; ; border:1px solid orange; font-weight: bold; ; background-color: #FFFFCC} .tx-pagebrowse li { display: inline; list-style: none; margin: 0 0; padding: 0 5px 0 0; } li.tx-pagebrowse-pages { width: 80%; } li.tx-pagebrowse-pages ol { display: inline; margin: 0 0; padding: 0 16px; } Clear cache - It works
For the German translation the following files must be there (translated):
/var/lib/typo3-dummy/typo3conf/I10n/de/mnogosearch/de.locallang_db.xml /var/lib/typo3-dummy/typo3conf/I10n/de/mnogosearch/pi1/de.locallang.xml /var/lib/typo3-dummy/typo3conf/I10n/de/pagebrowse/pi1/de.locallang.xml Take care that all folders and files have user:group www-data:www-data.
In order to have the pagebrowse navigation also on top of the results field add:
<div class="tx-mnogosearch-page-browser"> ###PAGE_BROWSER### </div> before line: <div id="tx-mnogosearch-results"> to the file: /var/lib/typo3-dummy/typo3/ext/mnogosearch/resources/template.html A reload in the web browser will show the change.
Highlight Search key words
In order to highlight the search key words more obvious, you have to edit the file ext/mnogosearch/resources/styles.css and replace .tx-mnogosearch-pi1-result-excerpt-selection to:
.tx-mnogosearch-pi1-result-excerpt-selection { font-weight:b; color:red !important; background-color:yellow; padding:0px 2px 0px 2px; } or just remove the comment: /* */ border: 1px dotted silver; margin: 0 3px; padding: 2px 2px; /* */ font-weight: bold;
Unfortunately when you click on the found page link, the search key words are not highlighted on this web page. In order to add that function you have to modify the PHP program ext/mnogosearch/model/class.tx_mnogosearch_model_results.php either local in /var/lib/typo3-dummy/ or global in /var/lib/typo3-dummy/typo3/. At about line 112 you add the line:
$result->url = $result->url."?&no_cache=1&sword_list[]=".$pObj->piVars['q'];
That adds the necessary commands to the URL and the search key words, as you can see in the last line of the search result.
In order to figure out in which PHP program the search word list is processed, you can search for:
rudi@rudi2u:/var/lib/typo3-dummy/typo3$ grepall sword_list ./ext/mnogosearch/model/class.tx_mnogosearch_model_results.php ./sysext/cms/tslib/class.tslib_search.php ./sysext/cms/tslib/class.tslib_pagegen.php ./sysext/cms/tslib/class.tslib_fe.php ./sysext/indexed_search/pi/class.tx_indexedsearch.php
grepall utility, for searching text in a folder.
#!/bin/sh # grepall utility, 2009-11-27 from Kofler Linux p.515 find -type f -exec grep -q $1 {} \; -print
If you click on the result link, the search word is highlighted and has a red border. In order to remove the red border search in file fileadmin/css/rudi-content.css for csc-sword and remove the text:
border:1px red solid;
depreciated error in TYPO3 version 4.3.x
If you click on a link of a search result you get an error message:
Function sql_regcase() is deprecated in /usr/share/typo3/typo3_src-4.3.3/typo3/sysext/cms/tslib/class.tslib_pagegen.php line 173
Solution is in: http://www.typo3.net/forum/list/list_post//97671/ and http://developer-services.net/?p=18
Add a line in /etc/typo3-dummy/localconf.php $TYPO3_CONF_VARS['SYS']['exceptionalErrors'] = 22527; Add a line in /etc/php5/apache2/php.ini date.timezone = Europe/Berlin $ sudo /etc/init.d/apache2 restart
mnogosearch debug / info
In order to debug mnogosearch you can use the -d command line option to show all setup parameters:
rudi@rudi2u:~$ sudo /var/lib/typo3-dummy/typo3/cli_dispatch.phpsh mnogosearch -v 3 -d # Defaults DBAddr mysql://root:password@localhost/mnogosearch/?dbmode=multi VarDir /var/lib/typo3-dummy/typo3temp/mnogosearch/var PopRankUseShowCnt yes PopRankUseTracking yes PopRankFeedBack yes PopRankSkipSameSite no Section body 1 8092 html Section title 2 256 text Section meta.keywords 3 64 text Section meta.description 4 64 text SaveSectionSize yes LocalCharset utf-8 BrowserCharset utf-8 #Allow *.htm *.html *.php *.txt */ *.pdf *.doc *.odt *.swx Mime message/rfc822 text/plain "cat $1" Mime application/msword text/plain "catdoc -a $1" Mime application/pdf text/plain "pdftotext -enc UTF-8 $1 -" Mime application/vnd.ms-excel text/html "xltohtml $1" Mime application/vnd.ms-powerpoint text/html "pptohtml $1" Mime application/vnd.oasis.opendocument.text text/plain "/usr/bin/php -q '/usr/share/typo3/typo3_src-4.2.12/typo3/ext/mnogosearch/cli/catoo.php' '$1'" Disallow *.rdf *.xml *.rss *.js *.css *.jpg *.png *.gif *.svg *.swf *.tif *.tiff Disallow *.bz2 *.gz *.zip *.xpi *.dmg *.exe *.scr *.avi *.jpeg *.ods *.psd *.rar Disallow *.fla Thumbs.db *.rpm *.rm *.qt *.mov *.pfb *.ttf* *.fon *.mpg *.mpeg *.ai Disallow *.t3x *.t3d *.dll *.dat *.cab *.inf AddType application/vnd.oasis.opendocument.text *.sxw *.odt AddType text/plain *.txt AddType text/html *.html *.htm AddType text/rtf *.rtf AddType application/pdf *.pdf AddType message/rfc822 *.eml AddType application/msword *.doc *.dot AddType application/vnd.ms-excel *.xls AddType application/vnd.ms-powerpoint *.ppt HoldBadHrefs 2d DetectClones yes HTTPHeader "X-TYPO3-mnogosearch: 2de8d32bea4c53eca8f5a3ca88951fbf" DefaultContentType "text/html" # uid=2 Period 10y Server http://rudiswiki.homelinux.org/typo3/
Disallow special file types
In order to disallow special file types like *.wmv or *.bmp you have to add that to a list in file ext/mnogosearch/cli/cli_mnogosearch.php either local in /var/lib/typo3-dummy/ or global in /var/lib/typo3-dummy/typo3/. Another option is to exclude printview files, with *.98.html.
Line 387 Disallow *.rdf *.xml *.rss *.js *.css *.jpg *.png *.gif *.svg *.swf *.tif *.tiff Append *.bmp # to exclude indexing of BMP pictures Line 389 Disallow *.fla Thumbs.db *.rpm *.rm *.qt *.mov *.pfb *.ttf* *.fon *.mpg *.mpeg *.ai Append: *.wmv # to exclude indexing of Windows Video files Line 390 Disallow *.t3x *.t3d *.dll *.dat *.cab *.inf Append: *.98.html
List of pages in this Category
-- RudolfReuter 2010-03-18 19:19:23
Go Back to CategoryTypo3 or FrontPage ; KontaktEmail (ContactEmail)