Generate Company URLs From a List of Company Names
How to query Google search results with SerpAPI to extract company websites by company names
Often in sales we have to make the most of imperfect data; which can lead to some headaches, but also some interesting solutions. At The Sales Factory, one of the common situations we run into is having to find contacts or companies from a list of only company names.
Why does having only company names make things more difficult?
Many companies can have the same or very similar names
Due to some companies having similar spelling, importing these companies into your CRM can lead to duplicates and potentially messy data
Some smaller or local companies may not exists on popular contact databases
This can cause you to completely miss some in your outreach
Inaccurate company matching in a contact database due to similar naming conventions can lead to unqualified contacts in your lists
What’s the solution to this?
In order to increase the accuracy of this matching process and to keep your CRM clean we need some form of a unique identifier. Company websites are the unique field we need in this case. Let’s look at how we can utilize the Google Search API from SerpAPI to get company websites!
How to perform a Google search?
First thing, make sure you have the Google search python package and load it into your environment. A link to it can be found here. Performing a search is actually quite simple and if at anytime you’re unsure of what some of the parameters are, the SerpAPI Playground is a great resource.
Performing a Google Search
There are even more parameters than what are shown here. These options should be good for the majority of cases. Now when you do your first search a lot of data will be returned. Let’s walk through exactly how we’ll grab the websites we need for a company list!
If you’re shocked at how easy that was, I was in the same boat the first time I ran a query!
Grabbing Websites from Company Names
The way we’ll be grabbing the company websites is through grabbing the first organic url from the Google search results. It’s important to mention that this method isn’t 100% foolproof. But, we can increase the accuracy by modifying the query to get the result we would like. Here are a few examples:
Company Name + Address/Location
Company Name + Industry
For a very niche example, let’s say you have a list of locations/cities instead of company names and need township websites for town/government employees. Your query in this case can be (without quotes): Location + “town website”.
Getting creative in this part will open up many more possibilities!
Why this first organic link?
This will be the first url that is an actual website result and not a sponsored post, company location info, etc…
How to grab this link
To get this url, we need to look for a specific field.
To make your life easier, here’s the code:
# Assuming you've created your parameters
search = GoogleSearch(params)
results = search.get_dict()
# First organic link
results["organic_results"][0]["link"]*Note: stick with the “link”, “displayed_link” is usually what’s visible on Google and in many cases not what you want for this use case.
Putting it all together!
Want the code for all of this so you can copy/paste instead of typing?
Load your packages, credentials, and list
Iterate through the company names in your list and extract the first organic link for each. Remember to modify your query for increased accuracy
You may have noticed I included an extra chunk in the for loop.
try: results["organic_results"]
except KeyError: results["organic_results"] = None
if results["organic_results"] == None:
df.loc[count, "website"] = "Manual Search"
else:
current_website = results["organic_results"][0]["link"]
# Insert website into df
df.loc[count, "website"] = current_websiteWhy is this added? This is because in some cases the query results in an empty search. There are other ways around this, but this is how I chose to do it. To make these companies identifiable in my list later, I set the website to a value I could easily locate and do some manual research.
Export your csv!
You’ve done it!
You’ve now created a new list that have urls based off company names!
This is a fairly basic example, but easily customizable to your specific use case. SerpAPI is something I have personally just started to dig into and the possibilities seem nearly endless!
If you come up with any interesting query parameters or SerpAPI use cases, feel free to leave a comment!