Author Topic: Technical note: Character set upgrade and unicode  (Read 456 times)


  • Administrator
  • Hero Member
  • *****
  • Posts: 793
  • Karma: 195
  • <admin>
    • View Profile
  • Decks
Technical note: Character set upgrade and unicode
« on: July 28, 2018, 06:56:21 pm »
When I created deckstats over ten years ago, supporting characters not in the latin alphabet (such as Cyrillic or Chinese) was not a high priority and the concept of emojis didn't even exist yet. Thus, I used the standard "ISO-8859" character set to represent text on all pages. In retrospect, this turned out to not have been the best decision. It forced me to build a lot of workarounds when I wanted to add support for non-latin characters later on. Over time, this has grown into an ever-more complicated mess of workarounds and workarounds.

Over the last year or so, I have been working towards finally cleaning all of this up and switching our entire site over to unicode (UTF-8). This has been a major effort requiring changes to large parts of our code on the backend.

I have already applied part of these changes back in January, and so far everything seems to be going fine. Thus, I am now planning to apply the final changes and complete the switch.

What does this mean for me?
In practice, there should be no visible changes whatsoever. The only difference will be that you may soon be able to use Emojis and other special characters in deck descriptions/notes etc. I will make another announcement when this is the case.

However, if you do notice anything weird happening, please let me know here or via email so that I can take a look!

Examples for things that might happen, and which I would particularly want to hear about:
  • Characters you have entered not showing up when you save text (eg. deck descriptions, names, cards, ...)
  • Characters you have entered being changed to other characters when you save them
  • Old text missing or not being displayed properly anymore
  • Problems with export/import of decks or collections
  • Javascript errors or missing images on the page
If you notice anything like this, please let me know immediately!

What about the forums?
The forums (and deck comments) are still stuck in a bit of a limbo, and will require some further work to upgrade to full UTF-8 support. More about this later!