Wesley's Tool-Box

The Tool-Box.info website has been running on the Apache - PHP - MySQL (APM) solution for the past 13 years. Each component has been constantly upgraded over the years, and recently I decided to update MySQL from 5.7 to 8.0. Once I managed to migrate the database to the new version, I discovered that all the Korean texts on the website came out broken. This was a sign of mismatched character set, so I looked for the exact cause.

First, I rolled back to 5.7 and checked what character sets were being used, using the following SQL query:

show variables like 'char%';

Sure enough, "character_set_database" and "character_set_server" were set to "latin1". Upon checking the database and the tables that contain the website data, their character sets were all set to "latin1_swedish_ci", the default choice. All the Korean texts were being saved to the database in Latin1 format from the very beginning. It got converted into UTF-8 as it was passed to the output, so it looked normal when viewed through a web page. But if you looked directly into the database, it came out broken. MySQL 8.0 apparently decided to output the text in its saved form, unlike 5.7, hence the problem.

The solution proposed by many of the Korean blogs that had a similar problem was to alter the character set of the affected databases and tables in the following manner:

ALTER DATABASE data_database CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

ALTER TABLE data_table DEFAULT CHARSET=utf8mb4;

ALTER TABLE data_table MODIFY COLUMN title VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

And also, add the following lines under [mysqld] in the MySQL configuration file (my.cnf):

collation-server = utf8_unicode_ci
character-set-server = utf8

Sadly, all this did not help one bit. Upon further analysis, I arrived at the conclusion that the underlying data was still in "broken" form even if the settings had the character set changed. The data itself has to be rewritten in UTF-8, so I needed to dump the database and reload it in the correct character set. First, the dump:

mysqldump -u root -p --default-character-set=latin1 data_database > dump.sql

The "default-character-set" flag was set to "latin1" to ensure that the data is dumped in its originally saved character set. In the dumped file, I changed all the "latin1" strings into "utf8mb4".

Now I simply had to restore it back, but the "Specified key was too long; max key length is 1000 bytes" error prevented me from restoring some of the tables. I tracked the problem down to the limitation of the MyISAM database type. Because the "VARCHAR" data type for a column needs 3 times more space for UTF-8 than Latin1, the character set change caused the key length to exceed the 1000-byte limit. With InnoDB database type, it was 3072 bytes by default since MySQL 5.7.7.

Because of this, I changed the database type mentioned in the file from MyISAM to InnoDB. So why was it set to MyISAM in the first place? It was because Full-text index was not available for InnoDB at the time of the database creation. It was enabled in 2011 with MySQL 5.6.

With both the database type and character set changed in the dump file, I restored the database like this:

mysql -u root -p --default-character-set=utf8mb4 data_database < dump.sql

I could now see that all the Korean text appeared correctly in the database. It would also look right on the website if I kept the changes to the my.cnf file mentioned earlier. Finally, I migrated the database to MySQL 8.0 again, and ran the "mysql_upgrade" command. Everything was working as intended, and I no longer needed the changes to the my.cnf, so those were removed.

Long story short, initial database settings from 13 years ago almost held me back from upgrading to the newest MySQL version, but all of them are now fixed.

The venerable Mac mini 2012, which took over the job of the server from the iMac 2008 in February 2013, showed signs of its age two months ago, refusing to boot due to corrupted Fusion Drive. I was able to remedy the problem, but I thought it may be a good time to move over to a new system. Seeing that Apple has not updated Mac mini in three years (and frankly, the 2014 edition was not an upgrade many had hoped for) I decided to return to using an iMac.

The iMac 21.5" 2017 was able to smoothly take over the Mac mini last month, but for some reason the system came equipped with macOS Sierra (10.12) instead of High Sierra (10.13) which was already a month old at the time. So I applied an extra caution and checked carefully that the apps I ran were compatible before manually upgrading. Finally, I made the switch to High Sierra today. It seems everything is functioning as expected.

One of the sore spots in using an Apple device (iPhone, iPad, and Apple Watch in particular) in Korea was the Apple Maps. Sure, you could use the natively developed map apps from the likes of Kakao or Naver, but regular apps using map function generally resort to the default Apple Maps data, leading to sub-par experience.

This had largely to do with the lack of map updates. When Apple Maps initially launched in September 2012, map data for Korea was sparse at best. It then received a major update in March 2014 that looked more complete at a first glance. However, delving into details revealed that the actual map data was from around latter half of 2012. This was clearly evident for Bitgaram City as you can see above. Roads weren't completed until 2013, and Apple Maps had much of the major roads missing.


Interestingly, there was another map update for Korea in April 2015. It showed all the major roads in Bitgaram City, as well the street of Juwol-ro in Gwangju that was completed in early 2015. This meant that the map was quite up to date at the time, but you could see it only if you were outside South Korea. The Korean server for the iOS Apple Maps that sends the data to users within the borders never received the update, leaving the Korean users with severely outdated map for several years. The screen caps shown here were made while I was on a trip to Mongolia a few months ago.

I actually asked Apple's technical support about this issue back in June. Sadly, no resolutions came out of this even though the staff did acknowledge the problem. Then, out of the blue, Apple Maps received yet another major update for South Korea yesterday afternoon. The new map data was fairly recent - judging from the building data, it seemed to be from early to mid 2017.


Speaking of which, yes, there were now outlines of most of the buildings. This didn't exist for South Korea before this update. The building data also contain height information, which enabled this nice flyover-style view of the map in 3D. With the updated road and building information, I felt that it finally became good enough for in-app uses, such as location-based arrangement of photos in the Photos app. With a few more feature additions and beefing up of POI data, it should be good enough for stand-alone uses as well.

Moon Jae-in became the 19th President of the Republic of Korea on the morning of May 10, 2017 immediately after the confirmation of his win in the snap election. Naturally, lots of books written by or related to him were flooding out to the bookstores to capitalize on the occasion. For my kids, I ordered this cartoon biography targeted for young readers (who? special - Moon Jae-in) online yesterday.

To my surprise, the publisher was very keen to have the book updated. The 1st print of this book only came out on March 7 of this year, a few days before the constitutional court upheld the impeachment of Park Geun-Hye. This meant that they were pretty confident of how the events would play out around when the National Assembly approved the motion to impeach last December and started to get the book ready. The 2nd print was dated May 18, which is today, implying that I got one of the first shipments of the even more updated version.


Indeed, this print of the book ends with Moon officially becoming the president. The 1st print had the same pictures but with a somewhat placeholder caption, so the publisher apparently anticipated the results and did a quick edit after the election was over. I have to say, I was quite impressed.

The most important election in the recent Korean history has started this morning - the 2017 Presidential Election. It's being held more than 7 months earlier because the citizens like us have successfully ousted Park Geun-Hye from power. We made our voices heard with various protests, and it resulted in her impeachment at the National Assembly; the decision was ultimately upheld by the constitutional court last March. Now our family put our efforts in electing a new president, who will hopefully clean up the giant mess and make new progress.

Early this morning, Park Geun-Hye has been arrested on multiple charges including bribery. She had been ousted from presidency just 3 weeks ago. Millions of people, including my kids as shown here, had been calling for this to happen for the last several months. Things are progressing - wish us luck.

Less than an hour ago, the Supreme Court approved the motion to impeach the Korean President Park Geun-Hye decided by the National Assembly on December 9, 2016. Much of the nation had been calling for her resignation or impeachment for several months due to the corruption and mishandling of national affairs. That included the rally and candlelight vigil held just a week prior to the National Assembly's motion, which saw more than two million people gathering (including my family) near Cheongwadae, the presidential palace in Seoul. We still have a long way to go, but at least we're moving forward. That's a good thing.