Operating system name and version: Centos5.5
Version of smsd: 3.1.11
Smsd installed from: sources
Name and model of a modem / phone: phone Sony Ericsson K550i
Interface: USB

Hi All

I am novice on using smsd. I am using the phone above to send out sms. Currently, I encounter 2 problems, which are

1. Chinese Character Support. How do I enable or configure in order to get chinese character support on send and receive sms? Note: English character is working fine. However Chinese character become funny character in outgoing folder when submitted by sendsms command.


2. How do I receive the sms by connecting USB phone cable? I have configured init = AT+CPMS="SM","SM","SM" in /etc/smsd.conf. But no luck.

Please advise.

Thanks

mfever wrote

1. Chinese Character Support. How do I enable or configure in order to get chinese character support on send and receive sms? Note: English character is working fine. However Chinese character become funny character in outgoing folder when submitted by sendsms command.

You could upgrade to 3.1.14. It has sendsms script which is slightly changed. Perhaps the script from 3.1.11 does not work properly in your system.

If the latest script still does not create correct message, show what you tried to send and show the output from hexdump -C < /var/spool/sms/outgoing/<message file>.

mfever wrote

2. How do I receive the sms by connecting USB phone cable? I have configured init = AT+CPMS="SM","SM","SM" in /etc/smsd.conf. But no luck.

Try with init = AT+CPMS="ME"

Hi Keke,

Thanks for your reply. I have upgraded to 3.1.14. I still can't send sms for chinese words. Below are sample I sent from terminal.

[root@localhost ~]# /usr/local/bin/sendsms 60120000015
Text: 测试手机
--
Text: 测试手机
To: 60120000015

File in /var/spool/sms/outgoing/send_oq4007 is
To: 60126925157
Alphabet: UCS

mK��bKg:

[root@localhost outgoing]# hexdump -C < send_oq4007 00000000 54 6f 3a 20 36 30 31 32 36 39 32 35 31 35 37 0a |To: 60126925157.| 00000010 41 6c 70 68 61 62 65 74 3a 20 55 43 53 0a 0a 6d |Alphabet: UCS..m| 00000020 4b 8b d5 62 4b 67 3a |K..bKg:| 00000027

Should I change Alphabet: UCS to others? What does UCS mean? Any way to send by UTF-8 encoding?

Please advise. Thanks

mfever wrote

Should I change Alphabet: UCS to others? What does UCS mean? Any way to send by UTF-8 encoding?

Those "funny characters" are shown, because the message body is coded using UCS2 ( = UCS ) alphabet. This is normal, and I do not see why you still can't send SMS. When you receive that SMS using a traditional handset, characters should be shown correctly. If they are not shown correctly, you have some kind of a problem with your handset. I tried this kind of a message, but my handset does support chinese characters. However, if I send this kind of a message to my another server, it is shown correctly.

It is not possible to send using UTF-8 alphabet. Internal alphabet of smstools is ISO-8859-15. This means that if whole message can be shown using ISO alphabet, it can be sent using GSM alphabet which is 7-bit character set. If some character is outside of ISO-8859-15 alphabet, whole message is converted to UCS2, which means 16-bit Unicode character set. With UCS2 all chinese characters can be shown.

In the receiving side you will need an eventhandler which converts incoming UCS2 coded messages to UTF-8. See this topic: [answered] Accentuated character conversion for more information.

If you still cannot receive messages, set loglevel = 7 in the global part smsd.conf and report_device_details = yes in the modem section. Then restart smsd, try to receive and show the log here.

Hi Keke,

Thanks for your explanation. Sorry to you, I made a mistake on my sentence. I can send out sms but it is funny characters. Actually, I am looking a solution for chinese character which is viewable by any handset via the smstool engine sending in and out.

Could you advise any things I can solve?

Thanks

mfever wrote

I can send out sms but it is funny characters. Actually, I am looking a solution for chinese character which is viewable by any handset via the smstool engine sending in and out.

Chinese characters are viewable only if handset has support for chinese characters. Many handsets do not have this support.

For example:

I send SMS from my server to my handset which is Nokia N95. Alphabet is UCS2 and message body is:
短信服務器工具

My N95 shows seven "box" characters like this: ▯▯▯▯▯▯▯

With N95 I send this message back to my server. The received message file contains:



This is because that server does not run any conversion. After the conversion with a script, the message body is:
短信服務器工具

This means that the message itself is correct, but N95 just cannot show it.

Hi Keke,

My phone is chinese character support, by sending phone to phone is fine. I am new is telecommunication. Is smstools going to fix this issue? Or is there any list of phone support UCS encoding as you knew?

Please advise..

Thanks

mfever wrote

My phone is chinese character support, by sending phone to phone is fine.

In the global part of smsd.conf add a definition:
store_received_pdu = 3

Then restart smsd and send a message containing chinese characters from your phone to your modem. Show the message file here, so I can check what kind it is.

mfever wrote

Is smstools going to fix this issue?

No, because this issue is not caused by smstools.

mfever wrote

Or is there any list of phone support UCS encoding as you knew?

No. Almost all phones support UCS2 encoding, but this is not the case. Some phones do not show chinese characters.

Hi Keke

As per request, the file's content show below:
From: <phone number>
From_TOA: 91 international, ISDN/telephone
From_SMSC: 60120000015
Sent: 10-11-07 22:00:36
Received: 10-11-07 22:06:16
Subject: GSM1
Modem: GSM1
IMSI: 502126370935193
Report: no
Alphabet: UCS2
Length: 2
PDU: 07910621000010F5040B910621965251F7000801117022006323046D4B8BD5

mK�

Note: The chinese word type from sony ecrison K550i is 测试.

Please advise. Thanks

The incoming file is ok.

In the post #3 you were sending four chinese characters: 6d4b, 8bd5, 624b and 673a. Hexdump shows that the file is correct.

If you want to ensure that smstools surely sends this kind of a message correctly, use the global settings sent = /var/spool/sms/sent and store_sent_pdu = 3. Restart smsd and send this kind of a message again. When you have the message file in sent directory, show it here.

You can also check the pdu using PDU Converter. Paste the PDU to the PDU Message Entry/Display area, and click Convert. You can then check the result.

I assume that the message is shown ok, which means that if it's still shown incorrectly in the phone, there is a problem with a modem, or with a Service Center. I think that the modem may cause this problem, because 16bit alphabet is very usual and it is very uncommon if Service Center cannot send 16bit messages properly. Perhaps you could try sending with another modem or phone.

Another member johnaa is also using Sony Ericsson phone, but different model, and in this case the phone does not return message id properly. Perhaps in your case the phone is buggy too, and it causes that 16bit messages are not sent correctly through the AT command interface.

Have you checked that what was presented in the previous post?

I'm currently preparing 3.1.15 to be released, and this version contains automatic internal Unicode conversions (UTF-8 <--> UCS2) which should work with Chinese (and any other) languages too. This means that external conversions in eventhandler and checkhandler will be no more required.

It might be interesting to know if you still have problems, before releasing 3.1.15.

As said, the incoming message in post #9 is okay, but looks strange because it is not converted from UCS2 to UTF-8. After the conversion the text is shown as expected.

If you want to, you could send a message like sent in post #3 to one of my servers. I can then check how it is delivered.

I show the phone number in the Private block:

Hidden private text.

Hi Keke,

Thanks your suggestion and good to hear that your new release. I have double checked sending and receiving Chinese character with eventhandler suggested from the post is running fine and good.

For sending, I didnt use checkhandler since it already do the conversion but receiving, it need eventhandler. Once your 3.1.15 release. I will upgrade to test as well.

Hi Keke,

I run test recently, I found that I have difficulty on storing human readable character in mysql database if I did the UTF-8 conversion in eventhandler.
I used cat < $2 |iconv -F UNICODEBIG -T UTF-8 for content capturing. Mysql is running UTF-8 encoding.

The case is the English character will appear as funny character in mysql. However, the Chinese or Chinese + English mix character message is working well during sending process.

Can you give any idea to solve the "intelligent conversion" for both type of message storing in mysql?

mfever wrote

I used cat < $2 |iconv -F UNICODEBIG -T UTF-8 for content capturing.

It's not correct. You are trying to convert the whole file, but only the message body should be converted.

In the message file there are two parts: "header part" and "message body". Header part is always using ISO-8859 character set. Message body starts after the first empty line. Message body is using the character set which is told by the Alphabet header.

Use the eventhandler code from this post. It will detect the alphabet, change the header and convert the message body properly.