Difference for 7.1 and 7.8 RTE

I run a webwiz forum in Japanese and ever since I upgraded to 7.8, there have been times when the characters will get changed up (garbled..) instead of getting saved properly as the Japanese characters that were entered.  The funny thing is that it's not all the characters, not even some, but just a few.  Quite sporadic really, but some characters will most definitely get messed up.

I'm wondering what has changed maybe in the parsing of the text, or the rich text editor.  It *never* happened with 7.1 (which I had up until lately) but now happens.

At the same time, I have also changed over from SQL2000 to MSDE, would this make a dif?  From what I understand, they are both the same engine so I'm doubting any problems with it but thought I'd check that too.

Any thoughts from anybody would be a great help.  If I can't solve this, then I'll have to roll back the forum, as right now it's not very helpful when you can't read some of the words in the post.

thanks

Thanks borg

Just to double confirm, there hasn't been any change in parsing of the characters then?

If it's the server or DB, do you have any suggestions?  I actually had to transfer the data from another SQL server to my server and they were using a different font encoding in SQL (SQL_Latin1_General_CP1_C1_AS) so I set mine as that so that they posts wouldn't get messed up.  Did I have to do that?  I would prefer to set it to UTF or Japanese really..

Thanks again Borg, another thought came to mind was to change the data type of the topic message. The funny thing is that the title for the post, and the forum name and description *never* get messed up.  I tried changing it to nText, but it didn't seem to make a dif.. any ideas in that direction of thinking?  Why would only the topic message get messed up but nothing else?  Are they posted to the DB differently or something?

Thanks again for your help on this.

I wanted to post the solution for this.

I have found that there is a difference in the way the post is getting formatted.  There is an extra check to get rid of long strings.  So if I were to just press "a" for like 50 times, the forum would put spaces in.  I think this is new to ver 7.8 (not sure about 7.7, I don't have it)

Anyways, in the Japanese world, we don't have spaces inbetween words which causes the above function to cause problems like putting spaces where we wouldn't want them, and at times cutting the 2 byte character in half, which makes it unreadable and garbled.

If Borg isn't into making the forum useable for 2byte characters, then no probs, but I think it'll help somebody out there who is running the forum with 2 byte chars.

On another note, I have finally figured out why when the titles are longer then 41 bytes why they get all messed up and make my forum go haywire (it places html code on the end of the title of the topic.. very strange)  That also is because the title is getting trimmed halfway through the 2 byte character.  This is crummy, so the only way to fix it is to test whether you have chopped up a 2 byte character and then delete that half a character to clean it up.

Hope this helps somebody out there.