Can we use NLP to help job seekers find jobs that are a better fit?
With the Great Resignation looming over 2022, more and more workers are also estimated to be on the job hunt. A record number of people have quit their jobs since the beginning of the COVID-19 pandemic, and while this trend does not seem to be slowing down any time soon, it is clear that this worker revolution also necessitates the need for appropriate tools to help these newly-displaced workers find more fulfilling jobs. That’s why I set out to build a cover letter generator as a quick weekend project.
One of the most tedious parts of applying for jobs is writing a cover letter. Despite how much job seekers would want to share how their skills and experience are a good fit for a role, it is often a very stressful and tiring experience to write multiple cover letters for each job and each employer that a job seeker is applying to. With this cover letter generator, I used the tools and models provided by OpenAI to create a web application that lessens that burden of effort, while still allowing job seekers to create customized cover letters that display their passion and excitement.
- Use GPT-3 models for text generation, including Curie, Ada, Babbage, and DaVinci, to create an OpenAI engine that generates a response from a given prompt.
- Collect user-generated input including company name, role applied for, contact person, applicant information, personal experience, job description, and passion to create a prompt that customizes cover letter responses.
- Deploy the model as a web application using Streamlit, and allow users to select the model based on compute and time constraints, as well as input their responses according the the roles they are applying for.
Before we dive in to the process of creating the app, let’s take a deeper look at GPT-3. Some time in 2021, OpenAI’s newest model, GPT-3, made waves in tech. However, the hype was not without its risks — from discussions focused on issues including data ethics, model management, and moderation. So what exactly is GPT-3 and what made it so popular?
Generative Pre-Trained Transformer 3 (GPT-3) is a deep learning natural language processing model that can be used to generate “human-like” text, among other tasks. It can also be used to write code, poetry, stories, and other prompt-based output. According to their paper, the OpenAI researchers state that GPT-3 is “an autoregressive language model with 175 billion parameters”, a scale that’s ten times more than any other language model. Using this massive model, that has been trained using both larger compute power and larger datasets, the researchers were able to achieve strong performance on multiple NLP tasks, including machine translation, question-answering, word unscrambling, using novel words in a sentence, and even performing 3-digit arithmetic. Most notably though, GPT-3 can be used to generate news article samples that human evaluators have struggled to differentiate and identify from true human-written news articles.
More specifically, GPT-3 is a family of language prediction models — it is an autoregressive language model that employs deep learning to produce human-like text. Briefly, a language model uses statistical and probabilistic techniques to predict the probability of certain words occurring in a given order in a sentence. This next word prediction feature of language models can be applied in areas including part-of-speech (PoS) tagging, machine translation, text classification, speech recognition, information retrieval, news article generation, and question answering to name a few. Each language model in the GPT-3 family contains a number of trainable parameters, although all of these models are based on the same transformer architecture. A lot of the excitement around GPT-3 is partly due to the huge corpus that it was trained on — CommonCrawl, WebText, Wikipedia data, and a number of books as well.
The main considerations I had for building a cover letter generator included creating an intuitive and easy-to-use web app, that didn’t require a lot of information or tedious interaction from users.
At current state, the web application allows you to enter job-specific information including the company, role, hiring manager, and job description. It also allows you to include personal information like your experience, passion, and interest in the job. Finally, the app allows you to select what GPT-3 model to use, which normally shouldn’t affect the results *too* much, as well as the number of tokens to use (which, from a user standpoint, could affect how long they want their cover letter to be).
I built the app using Streamlit and deployed it on Streamlit Cloud. I created textboxes where users can input specific information about the job, the company, and their personal experience. Then, I added toggles in a
submit_button on the Streamlit interface that allowed users to tweak the GPT-3 model if they so desired.
To start exploring GPT-3, I signed up for an account on OpenAI. OpenAI gives you a few free credits when you start, and I’ve found that these $18 credits were enough for my personal weekend hackathon app. Note, however, that these free credits expire in three months — so if you’re building a portfolio project, make sure to remember that users testing this app will only be able to do so freely while your credits last.
OpenAI has a robust “Getting Started” guide, that introduces the key concepts and potential tasks that GPT-3 can do. Namely, the OpenAI API allows users to use GPT-3 for virtually any task that involves understanding or creating natural language (and code, which is in their beta phase currently).
There are four main OpenAI GPT-3 models that users can access:
- Ada – “Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost.”
- Babbage – “Capable of straightforward tasks, very fast, and lower cost.”
- DaVinci – “Most capable GPT-3 model. Can do any task the other models can do, often with less context. In addition to responding to prompts, also supports inserting completions within text.”
- Curie – “Very capable, but faster and lower cost than Davinci.”
The prompt I used to create the cover letter is the following:
So, a sample prompt would look something like the following:
Cover Letter PromptWrite a cover letter to Technical Hiring Manager from Amber Teng for a Machine Learning Engineer job at Google. I have experience in natural language processing, fraud detection, statistical modeling, and machine learning algorithms. I am excited about the job because this role will allow me to work on technically challenging problems and create impactful solutions while working with an innovative team. I am passionate about solving problems at the intersection of technology and social good.
And the auto-generated cover letter response would look something like:
Auto-Generated Cover LetterDear Technical Hiring Manager,I am writing to apply for the Machine Learning Engineer job at Google. I have experience in natural language processing, fraud detection, statistical modeling, and machine learning algorithms. I am excited about the job because this role will allow me to work on technically challenging problems and create impactful solutions while working with an innovative team. I am passionate about solving problems at the intersection of technology and social good.I believe that my skills and experience make me a perfect fit for this job. I am confident that I can be an asset to the team and contribute to the success of Google.Thank you for your time and consideration.Sincerely,Amber Teng
Although the cover letter text isn’t perfect yet, and at this point it still does require a bit of input, I was excited to see that, according to a few of my friends who read the letter, it would be able to pass as a “legit” cover letter written by a human.
A few ways to expand this app that I want to explore in the future include the following:
- creating an option for users to input a PDF file of their resume so that they can automatically upload their job experience and current skills
- add an html reader or a website upload button that allows users to include the job description and the company automatically — this feature would ideally use Selenium for webscraping both the job description and position or role, as well as the company name
- finetune various GPT-3 models on “good” resumes, that have both been created by humans or data scientists who are applying for the same jobs, as well as vetted by human “experts” or annotators