EECS 485, Web Systems
Distributed Search Engine
A search engine over 3,062 Wikipedia pages, with a MapReduce-built inverted index served by parallel Flask index servers and ranked by tf-idf and PageRank.
What I built
- Built a search engine over 3,062 Wikipedia pages in a two-person team, using a six-job MapReduce pipeline to produce a tf-idf inverted index split into three segments.
- Implemented a Flask REST API index server per segment that ranks results by a weighted combination of tf-idf cosine similarity and PageRank.
- Wrote a search server that queries the three index servers in parallel and merges ranked results with a k-way heap merge.