Introduction

In this article I will show you how to filter text for bad words. In other words I want to do a profanity check on text. This could be used to restrict user inputs or censor texts. But more in the use cases below.

I know this topic is a bit controversial because it seems to be something that is used in upload filters or other censorship things we don’t want. But for me it’s okay to filter out obvious swear words. We only want to filter words that should appear nowhere. I want to perform something like the below example or display a warning.

Just piss off => Just ********

Use Cases

• Restrict users from publishing/uploading inappropriate text
• Censor text to make it child proof
• Display a warning that this text contains bad language

Approach

While I was searching for a good API solution I noticed that there are no suiting solutions from Google or Microsoft (at least I didn’t found one). So after having a look at different APIs from different providers I stumbled upon the bad words API from prompt API. The documentation is really simple and shows exactly what you need. The API seems to return information about the amount of bad words, where they are located and a censored variant of the text. This was exactly what I was looking for. So I had a look at the pricing. This is the essential information:

• Pay as you go: 0.0001$ per Request
• Free plan: 3,000 Requests per month (100 Requests daily)
• Gold plan: 300,000 Requests per month (10,000 Requests daily)
• Diamond plan: 1,500,000 Requests per month (50,000 Requests daily)

Have a look at the pricing yourself if you want further information.

In addition you don’t have to provide any personal information except an email address. This is something I really like because it makes it easier to try the API out.

All in all a good first impression. So I decided to give it a shot and try the API out. So far it seems to be what I need for this use case.

Test Request

In order to make a few test request I just had to sign in with my email and subscribe to the API with a free plan. Afterwards there is an API key in the documentation. There are also example requests to see what you have to do to make a successful request. In short you have to perform a POST request with the text in the body and the API key in the header. In the request URL you can specify a censor character which will be used to censor the text.

const myHeaders = new Headers();
myHeaders.append("apikey", APIkey);

fetch("https://api.promptapi.com/bad_words?censor_character=*", {
  headers: myHeaders,
  method: "POST",
  body: "Just piss off",
})

The request result contains several information which might be useful depending on your use case. You can see the text you passed in as the “content” value. The “censored_content” value contains the censored text, in this case censored by *. You also get the amount of bad words in “bad_words_total” and a list of bad words in the “bad_words_list”. This list is an array with an object in it for each bad word. It provides the bad word itself and information about the character position of the word.

{
  "content": "Just piss off",
  "bad_words_total": 1,
  "bad_words_list": [{
    "original": "piss off",
    "word": "pissoff",
    "deviations": 1,
    "info": 2,
    "start": 5,
    "end": 13,
    "replacedLen": 7
  }],
  "censored_content": "Just ********"
}

Implementation of the Use Case

To implement an example use case I decided to build a small Web App using React. You can have a look at this GitHub Repository where my implementation is provided if you are interested. Just follow the instructions in the README.md of the repository. Feel free to try it out.

Basically I started with implementing a Text Area input field. This will be used to provide the text I want to analyze. On the bottom right a Submit Button is placed. This button can be pressed to submit the text from the text input to the API. On the bottom left side I have some room for warnings and a loading indicator which shows that the request is currently pending and the user should wait.

For this example use case I decided to implement two handlings of the API response. In the left bottom an error occurs if the bad_words_total value is greater than 0 and otherwise a checkmark is displayed with a successful message. Also the text of the input area is overwritten by the censored text of the API response. You can see both behaviors in the provided gif. In a real implementation it might be sufficient to do one of these actions. But for demonstration purposes I guess it’s good to see both.

if (profanityResult.bad_words_total > 0) {
  return (
    <div className={classes.item}>
      <WarningIcon className={classes.warningIcon} />
      <div>There are bad words in your text</div>
    </div>
  );
}
if (profanityResult.bad_words_total === 0) {
  return (
    <div className={classes.item}>
      <CheckIcon className={classes.checkIcon} />
      <div>No bad words found</div>
    </div>
  );
}

Difficulties in using the API

One thing was actually a bit difficult and not well implemented in the API. It is no problem to pass text to the API and get results to it. But passing special characters like < or > seems to be not supported. So right now this API is only functional with normal alphabetical characters. Also new line characters (\n) are not supported. But this is not optimal with my textarea implementation because a textarea supports line breaks.

So I decided to remove the newline characters by hand before passing the text to the API. Because I want to add the newlines again after getting the result back it makes sense to replace the newline characters with some character combination that the user might not put into the textarea.

// remove newlines
const textWithoutNewLines = text.replaceAll(/(\r\n|\n|\r)/gm, "+#*#+");

// add newlines
const textWithNewLines = profanityResult.censored_content.replaceAll("+#*#+", "\n")

I know this can lead to unwanted behavior if the users actually writes “+#*#+” into the textarea. But for me it seems to be a workaround with is okay. Do you have other solutions for this problem in mind? Let me know.

I will also contact the provider of the API and will ask if it makes sense to adopt the API to support newlines and more/all special characters.

Summary

I tried out a lot of different spelling and all the words from some English swear blog post I found. And the results where really good. All swear words in American English are perfectly detected. For British and Australian English not all words are detected, but some of them aren’t even real swear words. It highly depends on the context then. For example the word rubbish can be used as an insult but that’s not necessarily the case. So it makes sense to not filter that out.

The Bad Words API is currently only available in English. But if that’s no problem to you I would highly recommend using this API. In terms of performance and reliability I have nothing to complain about and the results where also really solid. For my described use cases I am perfectly fine with the results. Do you have other APIs you would suggest to try out? Let me know!