Serp, a new open source technology has been designed to perform two major functions: fetching and ranking in listings of products from the SERP. Serp is developed by the small but impressive team of experts at MPC, who have been working for the last five years with a range of web analytics technology. I was delighted to discover this software as it has several unique features that set it apart from competing products.
Serp’s unique features are found in the integration of a number of open source technologies into its architecture. Some of these technologies include Apache Kafka, Apache Storm, Hadoop, Node.js, Python, DHT, XML, Google’s GeoLocation API, and a feature called Serp Routing. This article will take a look at each of these technologies in order to explain their unique functions and role in the architecture of Serp.
Serp uses Apache Kafka as its platform for processing streaming data. Apache Kafka has many great features such as the capability to process multiple streams simultaneously, which allows it to process 100% of a single request. This ability to stream multiple data sources creates an enormous processing capacity, which enables Serp to process an incredible amount of information in a relatively short period of time.
Apache Storm is a framework for developing distributed systems that make it easy to partition server processes into different tiers. It was originally designed as an Apache project, however, it has become very popular on its own and is used by many people in their own applications. Using Apache Storm in conjunction with Apache Kafka enables Serp to process 100% of all the requests in under one second. The parallel processing of multiple sources increases the throughput of a Serp application and enables it to rapidly process data from various sources.
Geoffrey Loeong, a colleague of mine at MPC has written a tool called Hadoop. Hadoop is an open source platform that makes it possible to run a range of tools including MapReduce, which is a general purpose parallel processing framework. It allows you to focus on your data rather than worrying about how to operate and maintain a cluster of servers. Hadoop has a general purpose interface that allows you to interact with its objects using commands, commonly known as “queries”. They allow you to manipulate Hadoop’s data structure and functions and effectively create “cascading” queries which can scale and work as needed.
Apache DHT (Distributed Hash Table) is a powerful networking component that can be used to distribute information to several parts of a network at the same time. It is an extremely effective way to distribute key-value pairs across a network. For the past few years, Apache DHT has been in use within the Google Web Services and has been improved and included with the current version of Serp.
A feature called Serp Routing is used by Geoffrey Loeong and several other members of the Serp team to route requests within serp api infrastructure. Routing allows a single Serp application to control a multi-client application. It is the newest addition to Serp’s architecture and offers many exciting advantages over many other similar technologies.
In conclusion, a brief look at each of the technologies mentioned above, the flexibility offered by Serp allows Serp to serve many applications in different ways, allowing you to design unique applications from the ground up. By implementing both of these technologies into your Serp application, you can achieve high performance, scalability, and simplicity. In my next article I will take a closer look at Serp’s unique features and how they enable a range of applications.