# indexOf does not match unicode string

**URL:** https://discourse.processing.org/t/indexof-does-not-match-unicode-string/31319
**Category:** Coding Questions
**Created:** [July 18, 2021, 1:53pm UTC](https://discourse.processing.org/t/indexof-does-not-match-unicode-string/31319 "2021-07-18T13:53:24Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![JoseMY](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josemy/32/2016_2.png) [@JoseMY](https://discourse.processing.org/u/JoseMY)
#### Post date: [July 18, 2021, 1:53pm UTC](https://discourse.processing.org/t/indexof-does-not-match-unicode-string/31319/1 "2021-07-18T13:53:24Z")

</div>

Hi!

I am reading a file called “Symbola605.txt” with loadStrings().  
Then I use .indexOf at each line in order to find a word that marks the beginning of a character list.

But it seems that .indexOf does not work with unicode files, even when combined with normalize(). Neither do includes() or match().

You can see the code with the attached file here:

> **[indexOf and loadString - Jose G Moya Y - OpenProcessing](https://openprocessing.org/sketch/1235341)**
>
> I am having trouble using indexOf inside a UTF16 string that comes from loadString

I will copy here the code – but remember, to see it working you have to click the url above, since this sketch needs a file.

```auto
// I'm trying to read the file "symbola605" from the line containing a word.
// but it seems indexOf does not work,
// and neither does indexof + normalize().
//

var SymbolaChars,body;
function preload() {
  SymbolaChars = loadStrings('Symbola605.txt', getSymbolaGroup);
}

// Finds the string "Symbola" inside the string
function getSymbolaGroup(){
	var needle="Symbola".toLowerCase().normalize();
	var out="";
	var S="";
	for(f=0;f<40;f++){
		S=SymbolaChars[f].toLowerCase().normalize();
		out+="["+int(f)+"] "+S;
		out+="<br/>"
		n=S.indexOf(needle);
		if (n<0) {
			out+="does not include '"+needle+"'";
		} else {
			out+="includes '"+needle+"' at "+n;
   	}
		out+="<br/>";
	}
	createP(out);
}

```

Returns:

> …  
> [22] symbola.ttf  
> does not include ‘symbola’  
> …  
> [26] symbola is not a merchandise.  
> does not include ‘symbola’  
> …  
> [30] symbola covers the following scripts and symbols supported by unicode:  
> does not include ‘symbola’  
> …  
> [38] character repertoire of symbola  
> does not include ‘symbola’  
> …  
> (and so on)

---

<div class="post-metadata">

### Author: ![KumuPaul](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/kumupaul/32/13072_2.png) [@KumuPaul](https://discourse.processing.org/u/KumuPaul)
#### Post date: [July 19, 2021, 8:20pm UTC](https://discourse.processing.org/t/indexof-does-not-match-unicode-string/31319/2 "2021-07-19T20:20:38Z")

</div>

The issue is that your file is encoded in UTF-16LE. The p5.js function loadStrings uses the [fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) under the hood, and the folks at WHATWG, in their infinite wisdom, decided that [UTF-8 should be good enough for everybody](https://fetch.spec.whatwg.org/#concept-body-package-data). You could [open a feature request on GitHub](https://github.com/processing/p5.js/issues), but the easiest solution would just be to re-encode your file as UTF-8 (here’s a quick command line if you are a linux/bsd based system: [linux - Using iconv to convert from UTF-16LE to UTF-8 - Stack Overflow](https://stackoverflow.com/questions/17287713/using-iconv-to-convert-from-utf-16le-to-utf-8)).

---

<div class="post-metadata">

### Author: ![JoseMY](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josemy/32/2016_2.png) [@JoseMY](https://discourse.processing.org/u/JoseMY)
#### Post date: [July 20, 2021, 12:00pm UTC](https://discourse.processing.org/t/indexof-does-not-match-unicode-string/31319/3 "2021-07-20T12:00:07Z")

</div>

Well, that’s what I did after many tries to use str(16) to convert from utf8 to utf16: i used notepad to convert the file.
