Lucene in Action

Jack Herrington, author of the fabulous Code Generation in Action, was the first to review Lucene in Action for Amazon. This is what he had to say:

This is a solid, well-written, introduction to Lucene and related technologies. The book starts with an introduction to the architecture of Lucene, replete with a simple sample application, then goes into an in-depth review of the indexing, searching and querying. XML and HTML indexing are also covered. As are performance issues. The last chapter covers related technologies and other implementations of Lucene into other languages. This is the best book I have seen on Lucene. It's an informative, fun read, that is worth the money if Lucene is central to your application.

Thanks Jack! []

... suitable for indexing. Listing 7.5 DocumentHandler using the PDFBox library to extract text from PDF ... to the index and provide better search results in the end. We store the meta-data in the following Fields...

..., all have different APIs. Listing 9.1 shows a command-line program that illustrates the indexing and searching API and its use. This program first indexes several documents with a sin- gle contents field. Following that, it runs a few searches against the generated index and prints out the search...

... the analysis phase but aren't encoded into the index. Unless specified otherwise, the type word...

... documents are indexed with similar phrases. One document with uses "the quick brown fox jumped over...

... methods to make our later code clearer, as shown in listing 5.3. We index two similar phrases in a field...

From a lucene-user e-mail list announcement:

 SearchBlox is a J2EE Search Component that delivers out-of-the-box search functionality for fast and easy implementation with your websites, applications, intranets and portals. SearchBlox uses the Lucene Search API and incorporates integrated HTTP/HTTPS and File System crawlers, support for various document formats including HTML, Word, PDF, PowerPoint and Excel, support for indexing and searching content in 18 languages and fully customizable search results, all controlled from a browser-based Admin Console. Main features in this release: ============================== - Support for Disk-based search index. Administrators can now choose where the index is held during operations, In-Memory or On-Disk - Preset filter: a pre-defined search query that will be automatically added to the end-user's search query - Indexing performance and stability improvements - Bug fixes SearchBlox is available as a Web Archive (WAR) and is deployable on any Servlet 2.3/JSP 1.2 compliant server. SearchBlox is also available as SearchBlox Server. The Server is an integrated application incorporating everything you need to run SearchBlox. The Server includes SearchBlox J2EE Component, the Jetty Application Server and the Java Runtime Environment (JRE) 1.4. With the SearchBlox Server, there are no additional software requirements to deploy SearchBlox. The SearchBlox FREE Edition is available free of charge and can index up to 1000 documents. The software can be downloaded from http://www.searchblox.com 

Robert Selvaraj of SearchBlox contributed a case study on SearchBlox to Lucene in Action, appearing in section 10.3. [Permalink]

... 1.2 1.4-final 1.3 1.2 (partial) 1.4 (partial) Compatible index Yes (1.2) Yes (1.4) Yes (1.3) Yes ... index format, and so on. With this said, let's look at each port, starting with CLucene....

Dion Almaer has just published the case study, I Love Lucene, he wrote for Lucene in Action on TheServerSide integration with Lucene. In this case study, Dion details how he architected the search feature used on the TSS site allowing the indexing and searching processes to be flexibly configured, as well as how the search feature was integrated into the web tier. [Permalink]

Erik will be presenting Lucene as part of his Java Gems: Ant, Tapestry, and Lucene seminar at the index.php">Greater Boston Chapter ACM on Saturday, April 9, 2005. If you're in the Boston area, this is a real bargain - visit the GBC/ACM site for more details on cost and registration. [Permalink]

We just received this unsolicited e-mail that made our day:

Excellent book. I purchased the book and used it to help deliver a knowledge management solution based upon the c# DotLucene. I found the material very well written and organized. Even though the code samples were in Java, I found it was easy to translate to c# due to the way you presented the concepts.

I was able to create a MS Content Management Server crawler to index 15,000 knowledge base articles in less than 15 minutes. The customer's previous indexer used Full-Text indexing for SQL Server and took well over 2 hours. Searches that used to take over 7 seconds now return in milliseconds, not to mention the fact that the search results are much more relevant than before. Needless to say, the customer is extremely happy.

Thanks to your book, I was able to get this project completed in 5 weeks. Prior to that, I had very little knowledge of search engines or content indexing. I hope to have the opportunity to use Lucene in future projects. If you decide to write more, I would love to see a "advanced" book that describes some more of the inner workings if Lucene. I am interested in learning more about things like term vector calculations and more advanced analysis concepts.

I have learned a great deal and look forward seeing more in the future. Great job and thanks!

We will definitely keep these suggestions in mind for the second edition of LIA. If you'd like to send us suggestions/comments/compliments/errata, please e-mail us. [Permalink]

... scenarios work. We've set up an index with 500 documents including a key field with values 1 through ... ("owner", "sue"))); } // ... } In addition to the test index, setUp defines an all-encompassing query ... the performance caveats to using filters; and, if you're reusing filters without changing the index, be sure...

... textual values for two HTML elements that we want to index: the document's title and body. Listing ... , so we per- form the necessary checks before adding title and body to the index. Because the DOM API...

... certainly use POI for that, if you need to index document meta-data, too. Listing 7.9 POI...

......

...As we've explored in several previous chapters, exposing QueryParser directly to end users can lead to confusion. If you're providing a web interface to search a Lucene index, you may want to consider using the nicely done JavaScript query constructor and validator in the Sandbox, originally written by fellow Lucene developer Kelvin Tan. The javascript Sandbox project includes a sample HTML file that mimics Google's advanced searching options, as shown in figure 8.13. The query constructor...

.../), a GNOME component for indexing and searching of all kinds of files, including pictures. Beagle...

...PyLucene is the most recent Lucene port; it's released under the MIT license and led by Andi Vajda, who also contributed Berkeley DbDirectory (see section 8.9) to the Lucene codebase. It began as an indexing and searching component of Chandler (described briefly in section 8.9), an extensible open-source PIM, but it was split into a separate project in June 2004. You can find PyLucene at http:// pylucene.osafoundation.org/. Technically speaking, PyLucene isn't a true port. Instead, it uses GNU...

index.html"> Erik created a multimedia (slides and audio) presentation introducing Lucene for JavaLobby's Expert Presentation Series. This presentation gives a high-level overview of what Lucene is and touches on important topics such as analysis. Because using Lucene's API is simple, there is enough code shown in the slides to build a rudimentary document search engine.

View the index.html">"Meet Lucene" presentation at JavaLobby

Update: Erik's project manager, Duane Gran, comments on his java.net blog. Feel free to go directly to the Rossetti Archive search mentioned by Duane. [Permalink]

Page 51 contains pseudocode with 2 mistakes. Instead of fsWriter.addIndexes( Directory[] {ramDir} ); It should be: fsWriter.addIndexes( new Directory[] {ramDir} ); Also, there is a line that contains: ramWriter.close(); This call should be moved up one line, to come before addIndexes call mentioned above. This close() call needs to execute first, in order for newly added documents to get flushed before index merge. Otherwise, not all documents added to that ramWriter will be added to fsWriter. [Permalink]