User:Fivesick/sandbox/DeepPeep

DeepPeep

DeepPeep is a search engine that not only lets users to access indexed databases much like Google search does but also allows access to databases in the Deep Web which do not pop up in regular search results. The purpose of DeepPeep is to extend user access to the Deep Web and obtain information and search results hidden from the surface.

DeepPeep originated at the University of Utah where a group of students Luciano Barbosa, Hoa Nguyen, Thanh Nguyen, and Ramesh Pinnamaneni, began the project overseen by Professor Jualiana Freire. The project generated worldwide interest and was sponsored by the University of Utah, and as a result gained a $243,000 grant from the National Science Foundation.

How it Works

Similar to Google, Yahoo, and other search engines, DeepPeep allows the users to type in a keyword and returns a list of links and databases with information regarding the keyword.

However, what separated DeepPeep and other search engines is that DeepPeep uses the ACHE crawler, HIerarchial Form Identification, Context-Aware Form Clustering and LabelEx to locate, analyze, and organize web forms to allow easy access to users.

ACHE Crawler

The ACHE Crawler is used to gather links and utilizes a learning strategy that increases the collection rate of links as these crawlers continue to search. What makes ACHE Crawler unique from other crawlers is that other crawlers are focused crawlers that gather Web pages that have specific properties or keywords. Ache Crawlers instead includes a page classifier which allows it to sort out irrelevant pages of a domain as well as a link classifier which ranks a link by its highest relevance to a topic. As a result, the ACHE Crawler first downloads web links that has the higher relevance and saves resources by not downloading irrelevant data.

HIerarchical Form Identification

In order to further eliminate irrelevant links and search results, DeepPeep uses the HIerarchical Form Identification (HIFI) framework that classifies links and search results based on the website's structure and content. Unlike other forms of classification which solely relies on the web form labels for organization, HIFI utilizes both the structure and content of the web form for classification. Utilizing these two classifiers, HIFI organizes the web forms in a hierarchical fashion which ranks the a web form's relevance to the target keyword.

Context-Aware Clustering

When there is no domain of interest or the domain specified has multiple types of definition, DeepPeep must separate the web form and cluster them into similar domains. The search engine uses Context-Aware Clustering(CAFC) to group similar links in the same domain by modeling the web form into sets of hyperlinks and using its context for comparison. Unlike other techniques that require complicated label extraction and manual pre-processing of web forms, CAFC clustering is done automatically and uses meta-data to handle web forms that are content rich and contain multiple attributes.

LabelEx

DeepPeep further extracts information called Meta-Data from these pages which allows for better ranking of links and databases with the use of LabelEx, an approach for automatic decomposition and extraction of meta-data. Meta-data is data from from web links that give information about other domains. LabelEx identifies the element-label mapping and uses the mapping to extract meta-data with accuracy unlike conventional approaches that used manually specific extraction rules.

Ranking

When the search results pop up after the user has input their keyword, DeepPeep ranks the links based on 3 features: term content, number of backlinks. and pagerank. Firstly, the term content is simply determined by the content of the web link and its relevance. Backlinks are hyperlinks or links that direct the user to a different website. Pageranks is the ranking of websites in search engine results and works by counting the amount and quality of links to website to determine its importance. Pagerank and back link information are obtained from outside sources such as Google, Yahoo, and Bing.

Advantages

Because DeepPeep allows the users to access databases that are not indexed, DeepPeep will be able to access content from the Deep Web. By using DeepPeep however, researchers will be able to gain more information more easily and in higher quantities. Search engines like DeepPeep will be able to accurately and quickly sift through the DeepWeb and be useful for large collections of data such a research for climate, finance, or government records.

Disadvantages

Although people may think having unrestricted access through the Deep Web is advantageous, the Deep Web contains dangerous and illegal content. Because it was difficult to access the Deep Web, drug dealing and other illegal activities are done over the Deep Web. Through DeepPeep, access to illegal content will be made too accessible.

Beta Launch

DeepPeep Beta was launched ( find the date) and only covered seven domains: auto, airfare, biology, book, hotel, job, and rental. Under these seven domains, DeepPeep offered access to 13,000 Web forms. One could access the website at deeppeep.org, but the website has been inactive after the beta version was taken down.