There are several categories of search engine software: Web search or full-text search (example: Lucene), database or structured data search (example: Dieselpoint), and mixed or enterprise search (example: Google Search Appliance). The largest web search engines such as Google and Yahoo! utilize tens or hundreds of thousands of computers to process billions of web pages and return results for thousands of searches per second. High volume of queries and text processing requires the software to run in highly distributed environment with high degree of redundancy. Modern search engines have the following main components:
Searching for text-based content in databases or other structured data formats (XML, CSV, etc.) presents some special challenges and opportunities which a number of specialized search engines resolve. Databases are slow when solving complex queries (with multiple logical or string matching arguments. Databases allow logical queries which full-text search doesn't (use of multi-field boolean logic for instance). There is no crawling necessary for a database since the data is already structured but it is often necessary to index the data in a more compact form designed to allow for faster search.
Database search engines were initially (and still usually are) included with major database software products. As such, they are usually called indexing engines. However, these indexing engines are relatively limited in their ability to customize indexing formats (compounding, normalization, transformation, transliteration, etc.) Usually they do not provide sophisticated data matching technology (string matching, boolean logic, algorithmic methods, search scripting, etc.).
In more advanced Database search systems relational databases are indexed by compounding multiple tables into a single table containing only the fields that need to be queried (or displayed in search results). The actual data matching engines can include any functions from basic string matching, normalization, transformation, Database search technology is heavily used by government database services, e-commerce companies, web advertising platforms, telecommunications service providers, etc.
Famous quotes containing the words search and/or engine:
“Adolescents are travelers, far from home with no native land, neither children nor adults. They are jet-setters who fly from one country to another with amazing speed. Sometimes they are four years old, an hour later they are twenty-five. They don’t really fit anywhere. There’s a yearning for place, a search for solid ground.”
—Mary Pipher (20th century)
“The machine unmakes the man. Now that the machine is perfect, the engineer is nobody. Every new step in improving the engine restricts one more act of the engineer,—unteaches him.”
—Ralph Waldo Emerson (1803–1882)