httpWebRequest UNICODE support

I’m trying yo use httpWebRequest with C10. When I send national characters to any WebService (converted to UTF-8) procedure sends them as ?
I already set ContentType as ‘application/json; charset=utf-8’
and I couldn’t find to declare User-Agent (Header parameter does not support Agent declaration)

Anybody has any solution for this problem?

Hi,

Sorry, but I think you’re going to have problems sending any Unicode via the standard Clarion templates.

We have accomplished it via our Chilkat wrapper, but that is when we’ve loaded the content from an external source.

I hope that helps,

Regards,

Andy
www.noyantis.com

Hi Andy,

Thank you very much for your guidance and kind attention.

I also used Nettalk and it is working properly.
As you explained, I’m trying to use standart method. It seems, Clarion has prefered old method of synchronized communication (httpWebRequest). New generation applications are using asynchronized communication (WebClient).

Hello All
I need to write the Unicode content to MySql database.
Unicode content come from webpage and it contains correct content but immediately when I get the content (in the Clarion), it is converted to ???.
With other words, I’m using LOC:MyVariable = GLO:UnicodeContent and in that moment I have only ???.
Question is - how to store Unicode content to MySql database (and to read it back, of course) witout converting it to ???
Thanks.

Hi Daniel,

content is not converted to ??? unless it is converted from unicode to ANSI, and there are “no matching characters”. So you need to understand where the converting is taking place, and what your original encoding is.

So, first things first. You incoming web data is not “unicode”. Unicode is a mapping, not an encoding. Your incoming data is encoded from the unicode mapping - specifically it’s encoded as utf-16, or more likely utf-8. Most of the web is utf-8 encoded, so I’m going to assume that’s what you are referring to here. (The distinction between utf-8 encoding, and unicode mapping, is important.)

Now Clarion doesn’t really know how to display utf-8, but that doesn’t stop you using it in a string field. (utf-16 is somewhat more complex - there you can use a string but NOT cstring field. utf-8 works with cStrings as well as strings.)

So when you say “I have ???” - I presume you are looking at the variable. If you are using say a MESSAGE function to do this then you see ??? - but the string is still utf-8. You can pass it through the driver to the database as utf-8.

BUT the database might not be utf-8 encoded. It might be utf-16 encoded, or ANSI encoded. So before going to the database you might need to convert the string to utf-8, or utf-16 or ANSI. (Use StringTheory for that.)

In fact StringTheory has a method, PEEKRAM, which lets you view the raw byte data in the string. Visually you can then see what encoding it is in. That can be useful to understand where the text is being “converted” so that you see it as ???.

There’s a OpenClarion webinar every Wednesday, at 8am Pacific Time, which is like a Clarion User Group. For multi-step questions like this, and understanding what is really happening, it’s useful be cause we can work through it with you step by step. See https://www.clarionlive.com for details.

Cheers
Bruce

1 Like

OK Thank you. Yes, now I understand that many things need to be synchronized… I need to know in what encoding is text encoded, what encoding use database (maybe also table) etc. It make lot of sense, so thank you again.