pocketpedia iphone search - diacritics

[talazem]

Hello, I have recently purchased bookpedia due to the existence of pocketpedia. Wonderful stuff. I'm having one problem, though, which is that much of my library is composed of transliterated titles, with diacritics in the title and author's name: mainly underdots, overscores, etc. I don't think any of the keyboards provided by apple can type them. I can *see* them fine (they render with the font used by the iphone), but i can't type them -- which makes searching for a title near impossible.

I don't know if this is a limitation of the iphone, or of the program, but is there any way to have the search ignore the diacritics, so that by my typing the letters without the diacritics will produce a search result of all glyphs that contain that letter in addition to the diacritic?

Thank you.

[Conor]

Thank you for the feedback. We perform the search using the built in iPhone SQLITE (we store our database in SQL format for fast access and low memory usage), and unfortunately there are limitation. One of them being that SQLITE searches distinguish between plain characters and their diacritic counterpart. It actually has a bug that goes even further in that although case insensitive for ASCII characters (letters stored in 7 bits) it is not for the rest. E.i. a search for 'æ' will not match 'Æ' as a search for 'a' does matches 'A'. We will continue to explore possibilities but do not despair I have a more useful piece of knowledge: hold down a letter on the keyboard for 2 seconds to get a list of possible diacritics versions.

Thank you for buying Bookpedia to support Pocketpedia.

P.S. On a related note, since you brought of diacritics, I realize Amazon searches using diacritics outside of the Japan locale are not possible in Pocketpedia but has been fixed for the next version.

[talazem]

I have a more useful piece of knowledge: hold down a letter on the keyboard for 2 seconds to get a list of possible diacritics versions.

Thanks for your response. Unfortunately, I already knew about that feature of the keyboard; the problem is that the particular diacritics I use -- overscore (a horizontal line above vowels) and underdots (a dot under certain consonants) are not part of a single one of the keyboards Apple ships.

If anybody knows how to make a keyboard for the iPhone the way we can make keyboards for OS X using Ukelele, please let us know. Also, in the future, if 'pedia is able to somehow find a work around for this problem, I'd love to hear it.

I was hoping since this type of universal search works on Bookpedia on the Mac, it would also work on the phone. Guess not.

Perhaps, is there a way for me to make mass changes in Bookpedia, to convert characters with diacritics into their equivalents without diacritics? That way, I could have an iPhone collection that I could sink with my phone that does NOT include the diacritics. This would be fine for my purposes of using it "on the go".

[Conor]

the problem is that the particular diacritics I use -- overscore (a horizontal line above vowels) and underdots (a dot under certain consonants) are not part of a single one of the keyboards Apple ships.

And I thought I was being clever with my keyboard tip. I didn't read your original post closely and overlooked the mention of the overscore and the underdot.

I was hoping since this type of universal search works on Bookpedia on the Mac, it would also work on the phone.

We were also hoping. We use the same SQLite on the Mac, but the iPhone version is more limited and gives and error when a diacritics insensitive search is attempted. An iPhone update might bring the ability to do so; we have put in a formal enhancement request with Apple.

Bookpedia does not have a find and replace but you can use the underlying SQL database directly to perform a find and replace. Using the program Terminal, in your utilities folder, paste these four lines one at a time (update the middle lines and add more of them for the appropriate find and replace, this is an example with underdot N).

Code: Select all

sqlite3 ~/Library/Application\ Support/Bookpedia/Database.pediadata
update zEntry set zTitle = replace(zTitle, 'ṇ', 'n');
update zEntry set zAuthor = replace(zAuthor, 'ṇ', 'n');
.exit

Before you start backup your data folder at "~/Library/Application Support/Bookpedia". You should always have a backup but via direct SQL access it only takes one command to wipe all your Bookpedia data, the backup is extremely important in this case.

[talazem]

we have put in a formal enhancement request with Apple.

That is good to hear.

Bookpedia does not have a find and replace but you can use the underlying SQL database directly to perform a find and replace

May I add my request for this feature in the future then?

Using the program Terminal, in your utilities folder, paste these four lines one at a time (update the middle lines and add more of them for the appropriate find and replace, this is an example with underdot N).

Thank you so much for this -- I really appreciate your top-notch support here! Unfortunately, the code didn't work. It seems I am able to type diacritics into Terminal.app, but not when I am in "sqlit3" mode, as it were -- it just gives me a beep and does not allow me to type glyphs with diacritics (though, again, I can do so before entering into sqlite3). Am I doing something wrong?

Thanks again for all your help!

[Conor]

it just gives me a beep and does not allow me to type glyphs

Excellent point, I did not test it out, but now I have. Use the hexadecimal value instead.

Code: Select all

update zEntry set zTitle = replace(zTitle, x'e1b987', 'n');

Use this website to find the appropriate value, listed under "UTF-8 (hex)". Notice that the X is on the outside of the single quotes in the SQL command to denote that it's a hexadecimal value.

May I add my request for this feature in the future then?

Consider it added.

I really appreciate your top-notch support here!

Glad I was able to help find a workaround. I appreciate your kind words.

[talazem]

Conor, I (finally) got around to implementing this. For those facing a similar issue, I found using the terminal too difficult, and instead used a Firefox plugin called SQLite Manager, which allows for diacritics to be typed in.

Anyhow, I used the custom fields to make two fields: Title_No_Diacaritics, and Author_No_Diacritics, and used the SQLite to first copy the titles and authors over to these fields, and then to strip them off the diacritics. As a result, I had the Title with diacritics, Author with diacritics, and then Title_No_Diacritics and Author_No_Diacritics without them. Perfect.

Until I went to the iPhone app, only to realize that the search on the PocketPedia does *not* search the custom fields, and as such does not give me results from my hard-earned non-diacritic entries.

I could, of course, switch the fields, so that my non-diacritic titles are in the default Title, etc., but it is counter intuitive (because the titles *do* in fact have diacritics), and poses problems for other usages of BookPedia.

May I request that we be able to search *all* fields, and not just the five default ones? Either as a preference, or as the default?

[Conor]

The reason the Pocketpedia only searches the five default fields is a due to the CPU, searching all the fields is a bit intensive for the iPod Touch and the iPhone. With large collections the program no longer feels responsive. But a preference could be a solution, I'll test it out see if I can include it with the next release if the speed is adequate.

For now you could flip the fields and use the custom fields or "original title" to hold the diacritic titles and allow you to search the non diacritics on the iPhone.

Thank you for the tip on the Firefox SQLite Manager plug-in, it looks useful and something that many users might be interested in.