Text extraction and manipulation

This String package is different from all the other packages. The String package is a collection of useful functions that will be used all the way through your code. A lot of the times you just need one String function. That's why we chose to show you a list of small examples of where the String package will work for you. Keep in mind, that some of the situations written below are not the fastest/smartest way to get there. 

Example 1

Let's say you have a database table filled with URL people send you from a form. These URL's were not checked on format. So you probably have urls with all kinds of formats. To ensure all urls have the same format, we will write a function that uses a number of String package functions to clean up the url. Let's start by creating a variable holding an ugly formatted url and output the original url:   

var original_url = "www.amAdEupUrL.cOM/"; // a really ugly formatted url

To be follow the progress we are making while cleaning the url, let's output this first stage. First we need to implement the System package to be able to use the System.print function to print the variable to the console. Keep in mind that the use-statements always have to be used before any other code.

use System;

And then after creating the variable, then we can print out the variable. 

System.print(original_url);

Now you will see, in the console, that the original url is printed out in its ugly format. Let's plan out which steps we will take to clean up the url to a clean tidy url format like "http://www.amadeupurl.com":

1. Transform the string to all lowercase letters.
2. Check if the url starts with "http://", else add it.
3. Check if the url contains "www.". If not, add it.
4. Check if the url has trailing characters. If so, remove them by executing step 5 and 6.
5. Retrieve the number of characters to determine the number of characters we want to keep of the url.
6. Use substring to remove the trailing character.

var url = String.toLower(original_url); // 1. Transform the string to all lowercase letters
System.print("Step1: " :: url);

// second, test if url starts with 'http://', if not, add it
if (!String.startsWith(url, "http://")) { // 2. Check if the url starts with "http://"
   url = String.replace(url, "www", "http://www"); // 3. If the url contains www, replace it with "http://www" 
}
System.print("Step2 and Step3: " :: url);

// If url ends with a '/', then remove it
if (String.endsWith(url, "/")) { // 4. Check if the url has trailing characters
   var org_length = String.length(url); // 5. Retrieve number of characters
   var endOfSubstring = org_length - 1; // Subtract the length with one, for removing the slash
   url = String.substring(url, 0, endOfSubstring); // 6. Use substring to remove the trailing character
}
System.print("Step3: " :: url); // print the cleaned up URL

Let's say you want to know how many times a letter appears inside a piece of text. This example shows how you can easily calculate this and even repeated for multiple letters. 

Again, let's plan this example. First we will generate some text so that we can use it to test our function. Then we create a row of every single letter from the alphabet. Then we retrieve the number of occurrences per letter and output this per letter.

1. Create a text for testing purposes
2. use String.allMatches to retrieve every instance of the given letter
3. Calculate the number of occurrences
4. Output the number of occurrences and the letter itself
5. Create a function to repeat the functionality
6. Create a list of every letter in the alphabet and pass it to the function
7. Execute the steps 2-4 for every letter

In step 1 we are using the String.toLower() function. This is a function from the String package. To be able to use this function, we need to add "use String;" to the top of the code.

In step 3 we are using the Collection.lengt() function. This is a function from the Collection package. Add "use Collection;" to the top of the code.

In Step 5 we use the System.print() function. Add "use String;" to the top of the code.

1. Create a text for testing purposes:

var testtext = String.toLower("Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi ultrices risus id tristique hendrerit. Nullam semper orci non ex pellentesque mattis. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Suspendisse non nisl non purus luctus lacinia et ut magna. Phasellus suscipit massa ante, vitae ullamcorper velit ultrices eget. Suspendisse a congue tellus. In hac habitasse platea dictumst.
Curabitur ut cursus nisl. Donec rhoncus lacus quis massa tincidunt congue. Vestibulum scelerisque pellentesque dui, eu elementum est euismod ac. Aenean risus lorem, dignissim vitae finibus ut, tempor et orci. Mauris ac lectus risus. Donec interdum suscipit ultrices. Donec ac tempus ante, placerat aliquam felis.
Donec feugiat aliquam quam, convallis efficitur erat sodales at. Donec nisi tortor, interdum a rhoncus vel, vehicula at ipsum. Suspendisse non auctor libero, sed dignissim turpis. Donec ipsum justo, iaculis semper neque vel, placerat condimentum sem. Phasellus pulvinar, massa ac aliquam malesuada, odio neque euismod urna, at sagittis mi ex non ligula. Nullam rutrum libero et facilisis dictum. Maecenas dictum tempor mauris, nec cursus nibh semper non. Praesent id nunc eget erat ornare tristique ac et purus.
Phasellus felis quam, bibendum et pulvinar vitae, ultrices consequat justo. Vestibulum placerat vulputate magna, laoreet tincidunt ante venenatis non. Sed turpis diam, iaculis sit amet ante vel, pretium fringilla sapien. Sed bibendum tortor vitae nunc vestibulum vehicula. Donec eleifend eros neque, a ullamcorper sapien interdum sit amet. Donec dolor tellus, finibus vel orci et, lacinia mattis lectus. Curabitur lectus turpis, commodo sed venenatis eget, aliquet quis est.
Nullam iaculis vehicula rhoncus. Duis laoreet egestas iaculis. Maecenas dapibus fermentum purus, eget consectetur diam interdum eu. Vivamus rhoncus vulputate scelerisque. Sed sed dapibus massa. Donec ornare aliquam accumsan. In hac habitasse platea dictumst."); // 1. Create a text for testing purposes.

2. use String.allMatches to retrieve every instance of the given letter. We are using a regex expression to do this. For example: "[a]" to retrieve all letters.

var numOfChars = String.allMatches(text, "["::char::"]", 5); // 2. use String.allMatches to retrieve every instance of the given letter

3. Calculate the number of occurrences

var chars = Collection.length(numOfChars); // 3. Calculate the number of occurrences

4. Output the number of occurrences

System.print(chars ::" times " :: char); // 4. Output the number of occurrences an the letter itself

5. Create a function to repeat the functionality

function printNumberOfCharacters(text, char) {
	var numOfChars = String.allMatches(text, "["::char::"]", 5); // 2. use String.allMatches to retrieve every instance of the given letter
	var chars = Collection.length(numOfChars); // 3. Calculate the number of occurrences
	System.print(chars ::" times " :: char); // 4. Output the number of occurrences an the letter itself
}

 6. Create a list of every letter in the alphabet and pass it to the function 

var tests = ["a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z"]; // An array with every letter of the alphabet
foreach(letter in tests) {
	printNumberOfCharacters(testtext, letter); // Per letter execute the calculation and output the result
}

For the complete code of this example, use the download link at the bottom for the attached robot.