We continue in the iSocialWeb blog, with the series of articles on automations using artificial intelligence. It is the turn of a Python script that will allow you to classify, obtain the search intent and identify the stage of the funnel in which the user is from a list of keywords.
You know how important search intent and entities are in SEO.
That’s why in this script, created by Álvaro Peña, you will be able to extract both from a list of keywords.
If this sounds like science fiction or a complicated task, we can tell you that you don’t need any programming skills. It is easy to use and anyone can use it.
We have even prepared a video to support the less experienced user.
Also, if you know how to program, feel free to change it to your liking and if you can mention our authorship. Remember: Good Karma is Good to everyone.
Automating artificial intelligence with Phyton
The first thing you should know is that the script will run on Google Colab.
A free Google tool that allows you to run code in the cloud.
Also, clarify that Python is a very simple programming language and the standard for this type of task.
Let’s see now the functions of the script:
Script functions
- Import keywords from keywords.csv file
- Detects entities.
- Classifies the user’s search intent.
- Determines the stage the user is at in the conversion funnel.
Well, as you can see, the code performs 4 tasks that otherwise could take hours.
A great idea, don’t you think?
Now let’s see the steps you need in order to take advantage of something so powerful.
Steps prior to the execution of the script:
In order to work with the script, you will need an OpenAI API key and a list of keywords in a CSV file called “keywords.csv”.
It is important that the file name is exactly “keywords.csv”.
This is the name that will be called within the Colab code and the first row should have a column called “keywords”, which will be the header and where the list of keywords will be found.
The script will run the CSV file and use OpenAI to sort the keywords.
It will then do a series of cleanups to format them properly.
Therefore, you need to have an API KEY and enter it into the Colab.
How to run Python code in the browser with Google Colab
The advantage of Colab, also known as Colaboratory, is that it allows us to share content programmed in Phyton and run the code easily, whether you are a student, data scientist or AI researcher.
Let’s take a look at the next steps:
- Save an archive copy of Google Colab from the File menu in your Drive.
- Install dependencies that Google Colab needs to run the program in your browser by running first cell.
- Import the file named keywords.csv with the template provided in this article into the folder located in the side menu. If you change the name, don’t forget to reference the new name in the Google Colab file.
- If it has been executed successfully, a green checkbox will be displayed. Do not proceed to the next step, until that check is displayed.
- Add OpenAI API key. If you don’t already have one, you can create OpenAI APIs for free.
- Make a batch with the total number of keywords.
- Create empty lists that we will use to incorporate information, once the words have been classified according to their intention and stage of the sales funnel.
- Run Prompt: “For the following list of words: return for each of the keywords the search intent [informational, transactional or navigational] and the stage of the conversion funnel [discovery, consideration or conversion]. There will be one word per line in the format: Keyword | Intent | Stage”.
- If you have been able to complete the various coding instructions, it will return a table of keywords sorted by search intent and stage in the sales funnel.
To help you understand the script better, here you can watch a video with a demo of the process:
This content is generated from the audio voiceover so it may contain errors.
(00:00) Very good to all and to all. Today from iSocialWeb, want to share a Script of Python that goes you to allow, from a listing of keywords, classify and obtain his intention of research and the stage of the user in which it finds inside the funnel. For this, here have an example.
(00:22) you Can see that have three columns: in the first, have the listing of keywords, for example: creatine, creatine grow, table of sinuca. And to continuation, have his intention that can be informacional, transactional, etcetera and the stage of the funnel in which they find , for example: discovery, conversion… Then, to make this go to use Google Colab.
(00:38) Goole Colab is a tool of Google that allows us execute code in the cloud. In this case, code Python, that is a very simple programming language and is basically the standard inside the sector for this type of tasks. By what if you are inexperienced, go to learn something since it is very easy to use because you go to see that only it is to include some data And is all programmed.
(01:04) And if you are more more advanced users , can modify it to your taste This is a Script generated by Álvaro Crag. You can modify it, improve it, share it, and have it all commented. What see here in green are commentaries and go preceded of a pad. This is not code, simply, is so that you read it and so that you comprise it. As you can see, it is all very detailed.
(01:20) If still like this, have doubts can contact by Twitter or in the own commentaries of this video and will try to see them and resolve them. Voucher, that go to need to work with this? You go to need a key of OpenAI and since, go to use GPT-3 to do the classification and go to need a listing of keywords. Here you have an example, simply need an archive .csv Called keywords. This is important.
(01:41) The archive has to call keywords.csv, because afterwards, is what go to call inside the code. You can change it, modify it and give him another name but would have to modify the code. Here you have the example in the line 14. Ensure you that this is correct or if it does not go you to give errors. And afterwards, inside the document keywords.
(02:02) csv, that the first row only goes to have a column, that goes to be yours list of keyword. The first row has to be the headed, has to be the header, that goes to call keywords. Same case, has to be this name, since further down we go to look for the field keyword inside the .csv.
(02:17) By what , ensure you that they coincide, do not go you to give errors. The majority of the basic errors in this type of Script, are by failures when appointing things, are failures tipográficos. So you do not break you the head and ensure you that all is correct. Under the headed keywords, simply include all your listing of keywords. Here you can do a Export of your favourite tool or can go adding the manually.
(02:38) To continuation, go you to explain the code, step by step. For the most veteran, this simply executes, read yours. csv And goes to execute a prompt optimised to classify the keywords and afterwards goes to do a series of #cleansing to give you the correct format. And now, for the most inexperienced, go to go explaining, step by step, what does.
(03:00) All Colab execute all the commandos by cells. If you see these elements of text are separated of these of here below All they have, up in the left upper corner, a play in which we execute the code and see that the rows are numbered. The code executes from above to down. In this first cell, would give to the Play and here, whenever you access a Colab that have not created you, goes you to inform that this has not been created by Google.
(03:26) In this case, it has been created by iSocialWeb and can allow you execute it, or can cancel and copy it to your Google Drive. You can exert it from this button, or going to Archive and saving in Google Drive. This will allow you have your own Colab and save the changes that execute . Here you can it execute without problems, but if you make some change, will not save . By what have it present.
(03:49) Once you execute the first cell and is ready, the dependencies, can go to the second cell. Here what go to do is to import the archive, a bookshop that allows us go up archives and go up our archive. Here it is where have to select archive keyword.csv That have created. To continuation, will import a series of bookshops, that also will allow us execute the code of continuation, and afterwards will read our csv to a Date Frame. A Date Frame is a type of archive,
(04:22) inside a type of variable, inside Python, that allows us use the bookshop Panda. This will see it very usually, already have to be accustomed and allows us manipulate .csv And data structured of form more comfortable.
(04:39) To continuation, and east is the only point that will have to change, will have to enter your API of OpenAI. This is deprived do not share it, and is what allows you spend credits inside your API and generate contents. To continuation, will do some tasks of management and structuring so that our code run perfectly. We go to do a counter of keywords and an index.
(05:00) This goes us to allow command the requests in batches. Instead of doing, if have 100 keywords, do 100 requests to OpenAI, do a bloc of 100 of 100 keywords, in a same request and save a pile of credits. To continuation, count the number of keywords not to spend us in the iteration of the same and go to create three lists.
(05:22) The first list is of the keywords that have, the second is the intention of research of the same and the third is the phase of the funnel or stage in which they find . To continuation go, from the lines 35 until almost the end, until the 75 go to iterate by all ours keywords doing batches and looking for his intention. For this go to create a batch until 100 keywords, go to separate each keyword by this separador.
(05:47) This is very usual, when you see prompts of OpenAI. This goes you to allow a jump of line and what goes to allow is to differentiate a keywords of the distinct keywords. To continuation have the prompt that go to command to OpenAi. This is the key of everything and here have copied it to you to a document to see it whole, since it is something long.
(06:11) Then, we go to do GPT-3, go to ask him for the following listing of words and here will #paste our listing of words. You see that we leave two spaces of jumps of line, to differentiate clearly the listing and here will include our batch of separate keywords all by a jump of line, other two jumps of line to separate well the listing and go to say that it #give for each one of the keywords the intention of research: informacional, transactional or navegacional, and the stage of the funnel of conversion: discovery, consideration or conversion. There will be a word by line
(06:38) with the format keyword, intention, stage. Separate, all they, by a vertical bar If we go back to our code, see that we select DaVinci 003, augment the number of tokens to 1500, since the listing can be big and maintain a temperature of 0.7. With these values, can play especially with the temperature. 0,7 it works quite well.
(07:01) And to the equal that play with these values can play with the prompt. Still like this, the format would not change it, since you will have to modify the code. That is to say, this format, afterwards is what go to use to generate our prompt categorizado, by what do not change it.
(07:19) you Can try modify the number of intentions of research or the phases of conversion, but in this case, I would not touch a lot, if we do not know very very what do. If you control of the subject, without problem. This is, to experience. Once you do the prompt, go to execute the prompt and afterwards go to formatearlo.
(07:42) This, basically, separates the keywords of the distinct elements that have asked. The keyword and stage, from his vertical bar, and the substring go them to go cleaning. And we go adding to our listing, the commando append, what does is: it adds at the end of the list, those that have gone pillando. That is to say, have our list of keywords, of intention and of stage that have created here up that are empty.
(08:05) Little by little we will go adding them, of one in one, each one of the intentions, keywords and stages. Here we go to go throwing the bloc, go to say that pille the 100 following and finally, go to create another Dates Frame with all these data. And here we would have the final table. Once you have done this, can download it, spend it to .csv And save it as you want to.
(08:27) So, with all this, would have a complete classification simply with two clicks. What takes in installing the dependencies, what take in dipping your API of OpenAI and what is to give him to click to this second code. This executes quite fast. Thank you very much By your attention and any doubt, are by here. A greeting.
Where do I find the script to rank the search intent of your keywords publish content by AI in wordpress?
You can access Alvaro’s script below:
We also leave here the CSV file to upload in the colab using our template:
What problems can I encounter when executing content scripting?
There are some problems that you may encounter when working with the programme that may prevent it from working properly. However, some of them can be solved by simply following the step-by-step process and carefully reviewing the video.
There are also external problems that may occur:
- CSV encoding: Sometimes, the CSV file is generated with a different encoding or contains elements that contaminate the document and make it unreadable. To solve this problem, you can use a demo file to test the tool.
- OpenAI’s availability problems: Sometimes OpenAI may crash and not return data, especially during US hours. This can cause a program crash, as it will not receive the necessary data in the request to the GPT AI.
What are entities in artificial intelligence?
Well, entities are objects or concepts that are used to describe information in content and are very relevant in the development of artificial intelligence applications.
For example, an entity can be a person, a place or a product.
Entities are used to identify and classify information in a corpus of data, which facilitates data processing and analysis by AI.
What is the relationship between entities, search intent and sales funnel?
There is an imply relationship between entities and sales funnel because both are essential for a successful SEO strategy.
Entities help search engines understand the content, contextualize search intent and establish the main topics of a web page.
On the other hand, the sales funnel helps companies understand where the user is in relation to their purchase decision and therefore helps define how to attract and retain potential customers.
Together, these two elements allow companies to improve their search engine rankings and increase online visibility.
Can the search intent of a keyword change?
The search intent of a keyword can change over time.
As John Muller confirmed to us: Yes, there are always changes in the way queries are treated (and how users expect them to be treated): the static “assignment” of an intent to a query is sometimes incorrect.
Yeah, there's always movement in how queries are treated (and how users expect them to be treated) -- statically "assigning" an intent to a query is bound to be wrong at times.
— johnmu is a ranking factor and so are you 🐀 (@JohnMu) October 18, 2022
Do you now understand the value of automating with python and taking advantage of AI in SEO?
Now, we are sure you do!
We hope you get the most out of our script and give us some love on social media.
So please, share this post with your friends and professional colleagues to help them and let us know if it has helped you!
References and bibliographical sources
Want to improve the performance of your sales funnel and increase the SEO relevance of your entities? We can help!