Author Topic: Restoring Japanese Text  (Read 4001 times)

0 Members and 1 Guest are viewing this topic.

Offline EBE@RTF

  • Team Leader - Australia
  • Administrator
  • Challenge Mode
  • ******
  • Posts: 2956
  • Rating: +14/-2
  • Gender: Male
  • Gigantic Ska Robot
    • Team RTF Australia
Restoring Japanese Text
« on: February 27, 2010, 11:37:42 PM »
One of the unfortunate side-effects of the recent server migration was (once again) the loss of all non-english language characters in all posts on this forum.

This is largely to do with the MySQL database the forum uses defaulting back to a non-UTF8 character set. UTF8 is a standard that handles characters from all major languages, but when you try to load UTF8 data into a latin database, you end up with a whole bunch of ?s.

I have corrected the database back to the UTF8 character set, which means everyone can start posting japanese characters again! However, this doesn't restore the japanese characters in old posts (because they have actually been replaced with a question mark in the database itself).

The solution to this is to reload a database from before the server migration. The problem with that? It was a month ago, meaning we'd lose all posts made since then.

The other solution, and the one I've decided to undertake, is to take the month old forum backup, and a backup of the current file, and manually locate and transpose all japanese characters that were lost. Since we are talking about a 10MB text file, that's going to take a bit of time.

I can use the fact that everything was replaced with ??s to my advantage a little, and I'm just searching through the file for consecutive ??s. This isn't perfect, since it won't catch individual ?s (which would take forever), and it won't restore any text from the last time we had this problem. But it should restore enough of the missing text, and all of the posts made in the 日本 forum to date.

What I will need to do when working on this project is shut down the forum, so no new posts are made while I'm editing the database dump. I'll try to do this at times when the forum isn't really visited much, but should you be greeted with the forum maintenance message, sorry, please try again later.
« Last Edit: February 27, 2010, 11:39:50 PM by EBE@RTF »
Team Ranking Task-Force
Part of Australia's DDR community since 1999

Offline ZDMKylie

  • RTF Team Member
  • Challenge Mode
  • *****
  • Posts: 816
  • Rating: +10/-0
Re: Restoring Japanese Text
« Reply #1 on: February 28, 2010, 11:39:40 AM »
You're awesome. That is all.


  • Guest
Restoring Japanese Text
« Reply #2 on: January 21, 2017, 04:37:23 AM »
I tride dragging them into their barracs but they are not restoring do you mean build a completely new unit and drag that one over the one that loast men?