Megi Sharikadze, a research manager at the Leibniz Supercomputing Centre in Munich, Germany, first became aware of the need for more specialised, local search engines during her quest to find the nearest post office. 

For several years, Google Maps was directing her to a post office quite a long way from her home. But as she decided to explore some more, she found the nearest post office branch much closer to home by searching directly on the German post website.

“It made me realise that there should be more specialised search engines that show all the details of local infrastructure,” says Sharikadze. 

She is now part of an EU-funded research initiative called OpenWebSearch.EU that is working to ensure more diversity in European internet search engines so that local search options can be more easily developed. 

The project derived from a bottom-up initiative in Germany which turned into a European effort, bringing together 14 renowned research and computing centres.

The idea behind the initiative is to change the current state of affairs, where non-European digital platforms control web search, creating a one-sided ecosystem that forces many users – including scientists, companies and the media – to optimise their content for those platforms.

“We want to empower communities so that they can easily filter and find those parts of the web that are relevant to them,” says Michael Granitzer, professor of data science at the University of Passau, Germany, and co-ordinator of OpenWebSearch.EU. 

One size doesn’t fit all

Search engines have become an integral part of everyday life over the last three decades. The problem is that the ability to search for information on the internet is almost entirely dominated by one company – Google. So much so that 'to google' has become a common verb in English – and many other languages. 

As of August 2024, Google is by far the world’s most used search engine, with a global market share of 90.5%. In 2024, Google’s dominance was ruled an illegal monopoly in a case brought by the US Department of Justice, while the EU’s top court also fined it for abusing its dominant position. 

 

'We want to empower communities so that they can easily filter and find those parts of the web that are relevant to them.' Professor Michael Granitzer, OpenWebSearch.EU

Some European search engines exist, but they mostly use search results from American companies like Google and Bing. 

The digital world is dominated by IT companies based in the USA. This means that on top of earning a lot of money, they also collect a lot of information and data about their users.

The EU’s response is the Digital Markets Act, which came into force in March this year, aiming to prevent these larger companies from using unfair practices to maintain an advantage in the digital market. That means technology startups, such as those producing search engines, will be able to compete without unfair conditions limiting their development.

Build it and they will come

The researchers at OpenWebSearch.EU aim to develop the core of a European Open Web Index (OWI) that developers could then use to build their own more tailored search engines. This will allow Europeans to enter the digital market with their own products, giving the European digital market a chance to grow.

An OWI is a digital infrastructure containing a set of data products in a structured form, which can be used by search engines that send information to the end user. 

One of the reasons for building a European OWI is that the EU has different data protection laws to the US, where Google is based. Having an OWI also means having infrastructure centred around European laws and values. Additionally, Europe has more language diversity and the OWI aims to support multiple languages.

“Crawling the web is a completely different thing in terms of legal settings in Europe than it is in the US,” says Prof Granitzer.

European boost

The OpenWebSearch.EU team includes 14 big European research centres that are leaders in this field. They started work in 2022 in a research effort that will run until August 2025. 

 

'By allowing a diverse set of web search engines to be created, we hope to counteract the current monopoly in the digital market and therefore create, again, more diverse information access.' 
Professor Michael Granitzer, OpenWebSearch.EU

The Leibniz Supercomputing Centre, where Sharikadze works, is a partner. Other partners include CERN – the home of the original index of WWW servers – and the Open Search Foundation, which Prof Granitzer credits with having come up with the idea for the project. 

The foundation, based in Starnberg near Munich, is dedicated to making the internet open and transparent in an attempt to boost European sovereignty in the digital market.

From niche to mainstream 

Prof Granitzer says that everybody will be able to use the OWI to create any kind of search engine they want. For example, one based entirely around Pokemon, or another specifically for children. 

Building a search engine requires access to resources that a lot of people do not have. The project partners believe that creating an index using European infrastructure will lower the entry barrier for those in Europe hoping to create new search engines.

There are three ongoing use cases developing a search engine with the OWI, including one that will function as a locator for restaurants in Slovenia and a search engine that combines web data with geoscience data, which Prof Granitzer says could be useful for environmental scientists.

“By allowing a diverse set of web search engines to be created, we hope to counteract the current monopoly in the digital market and therefore create, again, more diverse information access,” he says.

Author: Jack McGovan. This article first appeared in Horizon magazine.