|
|
| SMS Server Tools 3 Community |
| Welcome, Guest. Please login or register. |
Thu Apr 18, 2024 21:44 |
Login and Post Reply
Page: 1 2 Next
| Author |
Post |
|
|
#1 Mon Mar 07, 2011 14:21, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
|
Operating system name and version: Debian GNU/Linux squeeze/sid (2.6.32-3-686) Version of smsd: 3.1.11 Smsd installed from: package repository Name and model of a modem / phone: ZTE K3565-Z Interface: USB Hi everybody, I'm trying to send a bunch of SMS over several usbserial modems. I'm trying to send all sms in fast sms sending mode so I've put at+cmss=2 as init string. When I use one modem at a time and use global spool (/var/spool/sms/outgoing) everything goes well and it takes aproximately 0.8 - 0.9s per SMS, which is ok. But when I try to send over two modems where I have a queue for each of them - the speed goes normal (not fast) and it takes about 2.5s per SMS (which is intolerably slow in my case). When I start two separated smsd instances, each having separate config and pid file, then it goes fast again. So is there a chance to make sending over queues to go in fast speed mode? Here is my global config file: devices = GSM1,GSM2
outgoing = /var/spool/sms/outgoing
checked = /var/spool/sms/checked
incoming = /var/spool/sms/incoming
logfile = /var/log/smstools/smsd.log
infofile = /var/run/smstools/smsd.working
pidfile = /var/run/smstools/smsd.pid
failed = /var/spool/sms/failed
sent = /var/spool/sms/sent
stats = /var/log/smstools/smsd_stats
receive_before_send = no
autosplit = 3
max_continuous_sending = 7200
[queues]
GSM1 = /var/spool/sms/GSM1
GSM2 = /var/spool/sms/GSM2
[GSM1]
device = /dev/ttyUSB3
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
rtscts = yes
cs_convert = yes
report = no
sending_disabled = no
queues = GSM1
[GSM2]
device = /dev/ttyUSB8
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
rtscts = yes
cs_convert = yes
report = no
sending_disabled = no
queues = GSM2
'smsdconf' Syntax Highlight powered by GeSHi Also, is there a chance to force multiple modems to serve one queue? (hundreds of modems). Any help will be appreciated  Best regards, Ljubomir
|
|
|
#2 Mon Mar 07, 2011 19:27, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
First, you need to upgrade to 3.1.12 or later. Debian has migrated 3.1.14-1 to testing today, so it's available from the repository.
In the 3.1.11 there is a bug when more than one modems are serving the same queue. Lock files are not checked by default, and this may cause slow performance in some cases. A modem setting trust_spool = no helps for this (with 3.1.11), but there is then a one second delay when searching for a file. This probably is too much for you.
By default all modems will serve the same queue, which is a "checked" directory, when queues are not defined.
You have had headers "Queue: GSM1" or "Queue: GSM2" in SMS files, because without this definition all messages are targeted to GSM1. It may be possible that other modem had nothing to send for few seconds, because some number of messages were spooled to another modem. This causes that the radio link is closed (typically after 3 seconds of idle time), and benefit of AT+CMMS is lost at that time.
It could be useful to see some pieces of a log, but first upgrade to 3.1.14 which provides more infomation about spooling.
Some settings probably can be used to speed up the spooling, but before I recommend anything, I need to know how do you exactly create outgoing SMS files, and what is the number of files in outgoing directory as a peak?
|
|
|
#3 Tue Mar 08, 2011 11:13, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Hi Keke, thanx for fast reply ! I guess that problem is actually more profound than I thought. The speed is always normal, never really gets to fast mode: I only measured the speed of emptying outgoing directory. So, didn't realize that all messages are moved to checked or some other queue and than being processed. Anyway, I upgraded to 3.1.14. The speed I'm trying to get is definitely achieved with this modem by hand: I've made simple script that just triggers at command for sms send, waits for OK and so on ... Am I wrong or this starts to look like feature request? Simply what has to be done is to invoke at+cmss=2 seconds before sending is started. I put keep_open = no (hoping that init string will be sent to modem in apropriate time, but this didn't help as well). Here is a log of following configuration: - Two modems - Two separate queues, each modem serving one queue - Copied 3 messages in each queue at (almost) same time |
|
|
#4 Tue Mar 08, 2011 12:05, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
If you are sure that "More Messages to Send" works in your GSM network, we could patch the code. In this case you should download the latest package and prepare to compile it. You will also need development version of libmm. If you can do this, I will show later how to change the code. It is known that CMMS is not supported by many modems, and also it's not supported by many networks. But for those users who have the support, this feature might be very valuable. It looks like I'm going to include a modem setting "fast_mode" in the next version.  But let's test it first... |
|
|
#5 Tue Mar 08, 2011 12:08, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Deal ! I'm waiting for further instructions, now going to grab the source code .... |
|
|
#6 Tue Mar 08, 2011 12:15, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
Okay. Edit the file smsd.c, search the following and change the highlighted line: // Compose the modem command
make_pdu(to,text,textlen,alphabet,flash,report,with_udh,udh_data,DEVICE.mode,pdu,validity, replace_msg, system_msg, to_type, smsc);
if (strcasecmp(DEVICE.mode,"old")==0)
sprintf(command,"AT+CMGS=%i\r",(int)strlen(pdu)/2);
else
sprintf(command,"AT%s+CMMS=2;+CMGS=%i\r", (DEVICE.verify_pdu)? "E1" : "", (int)strlen(pdu)/2-1); // 3.1.4: verify_pdu
'c' Syntax Highlight powered by GeSHi |
|
|
#7 Tue Mar 08, 2011 12:32, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Hmmm, still same performances  Is it possible that spool engine takes to much time to feed the modem? (didn't read the source actually, just coming up with the idea ....) |
|
|
#8 Tue Mar 08, 2011 12:39, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
From the log we should see how much the time is spent between each SMS.
About your testing script, does it generate timestamps which will show the difference?
|
|
|
#9 Tue Mar 08, 2011 13:09, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
Another change, it may help but I'm not sure. File smsd.c: else
{
// 3.1:
if (0) // if (DEVICE.sending_disabled == 0 && DEVICE.check_network)
{
switch (wait_network_registration(1, 100))
{
case -1:
STATISTICS->usage_s+=time(0)-start_time;
flush_smart_logging();
return 1;
case -2:
STATISTICS->usage_s+=time(0)-start_time;
flush_smart_logging();
return 3;
}
}
}
// Compose the modem command
'c' Syntax Highlight powered by GeSHi |
|
|
#10 Tue Mar 08, 2011 13:18, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Hi, here are logs and comparation. First, here is log of pathced smsd: Now, here is a script I'm testing cmms with: So as you can see the message is stored into memory and it is simply resent every time. I tried with text and pdu message, the results are completely the same (regarding the speed). Here are timestamps: The script is quite unstable and unuseful, I just made it for test as you can see |
|
|
#11 Tue Mar 08, 2011 13:24, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Ah I now saw new reply from you When I said "patched" I meant on first one, not second one .... Be online back in a few hours and then I'll let you know if second one helped |
|
|
#12 Tue Mar 08, 2011 14:09, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
While waiting for the new log...
We are now talking about two completely different features. In your first post you were talking about AT+CMSS, but in the configuration AT+CMMS was used. I was thinking that the CMSS is a typo, as this feature is not supported by smstools, and even if it is implemented, it does not belong to the init string.
CMMS may work, but it depends on the network, as said. I have one more patch available, but let's see the current result first.
CMSS will be problematic, if incoming messages are read. Stored messages should be in different memory, otherwise smsd will read them out and mark as "unsupported messages". To get this functionality work well, smsd should create stored message, send it from memory to multiple recipients, and then delete message. Lot of new functionality and testing will be needed. I think that if "lightning fast" speed is required for sending, external script like yours could be used.
|
|
|
#13 Tue Mar 08, 2011 22:13, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Hi again, I would say that with both patches the results are quite the same (new log is included). But reported time of sending is 1s, it actually fits performances provided by at+cmms. But it remains unresolved where smsd spends it's "spare time", because resulting time of sending is a little bit longer (2 seconds per SMS, maybe more). |
|
|
#14 Wed Mar 09, 2011 09:59, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
ljubex wrote I would say that with both patches the results are quite the same (new log is included).
But reported time of sending is 1s, it actually fits performances provided by at+cmms. But it remains unresolved where smsd spends it's "spare time", because resulting time of sending is a little bit longer (2 seconds per SMS, maybe more).
Sending time is now better because network registration is not checked after the first message was sent. With "at+cmms" you meant at+cm ss, which is used in your script, right? Because those are two different things, results cannot be directly compared. Make the following changes: In the global part of smsd.conf, define logtime_us = yes. The next log will show better time resolution. In the file extras.c, apply the following change: if (!found)
*filename = 0;
else
{
strcpy(filename, fname);
// 3.1.12:
i = (int)(time_usec() - start_time) / 100000;
// if (i > 10)
writelogfile((i >= 50)? LOG_NOTICE : LOG_DEBUG, 0, "Took %.1f seconds to get a file %s, lost %i times, %i SMS files and %i LOCK files seen.", (double)i / 10, fname, lost_count, files_count, locked_count);
}
#ifdef DEBUGMSG
printf("## result for dir %s: %s\n\n", dir, filename);
#endif
return found;
}
'c' Syntax Highlight powered by GeSHi After those changes do the test again and show the log. Then, remove the patch shown in post #6 (file smsd.c): // Compose the modem command
make_pdu(to,text,textlen,alphabet,flash,report,with_udh,udh_data,DEVICE.mode,pdu,validity, replace_msg, system_msg, to_type, smsc);
if (strcasecmp(DEVICE.mode,"old")==0)
sprintf(command,"AT+CMGS=%i\r",(int)strlen(pdu)/2);
else
sprintf(command,"AT%s+CMGS=%i\r", (DEVICE.verify_pdu)? "E1" : "", (int)strlen(pdu)/2-1); // 3.1.4: verify_pdu
'c' Syntax Highlight powered by GeSHi Add a new patch in smsd.c: // 3.1beta7: Now logged only if a message file contained Report:yes.
if (report == 1 && !DEVICE.incoming)
writelogfile(LOG_NOTICE, 0, "Cannot receive status report because receiving is disabled");
if ((quick==0 || (*smsc && !DEVICE.smsc_pdu)) && DEVICE.sending_disabled == 0)
{
int i;
i = initialize_modem_sending(smsc);
if (i)
{
STATISTICS->usage_s+=time(0)-start_time;
flush_smart_logging();
return (i == 7)? 3 : 1;
}
put_command("AT+CMMS=2\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);
}
else
'c' Syntax Highlight powered by GeSHi This will show the execution time of AT+CMMS, and if it has remarkable effect when it was used together with AT+CMGS, every time when SMS was sent. Run test and show this log too. |
|
|
#15 Wed Mar 09, 2011 10:12, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
I'm on my way ,,,, I've just enabled miliseconds log, few minutes ago And regarding cmms/cmss: Script is using at+cmss (send mesage from storage) because there was manualy stored pdu on the modem, so it simply invokes mesage sending from storage. And at+cmms=2 (more messages to send) is invoked manually before sending. Also, when it is set to 2, there is no need to trigger it every time: only first message takes normal time, all of the following ones are going fast. |
|
|
#16 Wed Mar 09, 2011 10:31, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
And here is the log according to latest instructions: |
|
|
#17 Wed Mar 09, 2011 10:51, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Okay, so as far as I can see, there are two main reasons for difference in speed:
- First is this 0.5s acquiring of next message from spool
- And second one is two step scenario: smsd is waiting for ">" before submitting pdp string: in my scenario message is transfered once and sending is triggered with no reason to wait for modem once again
|
|
|
#18 Wed Mar 09, 2011 11:05, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
In many at command references I saw that at+cmgs is documented as "single step" command, meaning that <CR> character and PDU are supplied together with at+cmgs. That would save 300ms in this particular case. So, 500ms of message preparation would be the last remaining difference in performances I guess.
|
|
|
#19 Wed Mar 09, 2011 19:02, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
ljubex wrote And regarding cmms/cmss:
Script is using at+cmss (send mesage from storage) because there was manualy stored pdu on the modem, so it simply invokes mesage sending from storage.
Yes of course, but "at+cmss=2 as init string" sounds no good . It's not supported by smstools. ljubex wrote And at+cmms=2 (more messages to send) is invoked manually before sending. Also, when it is set to 2, there is no need to trigger it every time: only first message takes normal time, all of the following ones are going fast.
Yes, the manual says so. However, your log says that after +CMMS=2 was moved to the first time initialization, sending time was always 2 sec (~1500 ms). Before that it was reported as 1 sec. I hope that you can do two more tests, now when logtime_ms is enabled: First, change the put_command("AT+CMMS=2\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);to put_command("AT+CMMS=0\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);. I would like to see what is the sending time when CMMS is disabled. Second, remove that CMMS= command line (comment it out with //), and apply the patch in post #6 again. I would like to see what is the exact sending time when CMMS is enable during every sending. Please do these two tests first, before changing anything else. ljubex wrote Okay, so as far as I can see, there are two main reasons for difference in speed:
- First is this 0.5s acquiring of next message from spool
If you have a modem setting trust_spool = no in the configuration, remove it. I recommended to use it with 3.1.11 only, because of a bug in that version. ljubex wrote - And second one is two step scenario: smsd is waiting for ">" before submitting pdp string: in my scenario message is transfered once and sending is triggered with no reason to wait for modem once again
ljubex wrote In many at command references I saw that at+cmgs is documented as "single step" command, meaning that <CR> character and PDU are supplied together with at+cmgs. That would save 300ms in this particular case. So, 500ms of message preparation would be the last remaining difference in performances I guess.
Can you give a link to a manual which documents +CMGS like this? With a quick search and view I did not find any. All manuals said that it is necessary to wait the prompt. If some modem will work as "single step", it would be very interesting and likely I will include this feature in the next version. Hopefully your modem supports this, so I can use you to test the change to the code . |
|
|
#20 Thu Mar 10, 2011 09:51, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Ah sorry, now I saw a typo in post It's ok in my smsd.conf I'll have test results soon, and yes, there is a line: trust_spool = no so I'll test without it. |
|
|
#21 Thu Mar 10, 2011 10:26, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
First scenario (cmms disabled completely): Second scenario (cmms set every time): The difference is pretty obvious even by naked eye And regarding cmgs, take a look into this manual for ZTE modem family: http://download.maritex.com.pl/pdfs/wi/MG2636at.pdf Here, and on many other manuals I saw cmgs documented as: AT+CMGS=<length><CR> <pdu><Ctrl-Z/ESC> So, only important thing is <CR> in a string. I believe that ">" is sent unsyncronously from modem side, and input buffer of a modem is not affected. But of course this looks like strongly modem-dependent issue. I have several different types of modems with me, when I perform tests with ">" on all of them I'll come up with information. Meanwhile ... is there a way to store a bunch of pdu's on modem (in order to save time), and later to trigger sending, message by message? I mean not by cmgw but with some burst-like command? « Last edit by ljubex on Thu Mar 10, 2011 10:27, 159 months ago. » |
|
|
#22 Thu Mar 10, 2011 11:01, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Why it takes ~200ms between these two log points?
2011-03-10 11:03:54.472,6, GSM1: Sending SMS from to 38163440734
2011-03-10 11:03:54.672,7, GSM1: -> AT+CMMS=2;+CMGS=26
It's from point where smsd says "I'm about to send command" to "I've sent command", right?
|
|
|
#23 Thu Mar 10, 2011 12:19, 159 months ago.
|
|
Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
|
ljubex wrote The difference is pretty obvious even by naked eye
Yes, CMMS works as expected and there is no reason to set it every time. I will do those changes, as well as changes to "check registration" to the code. This feature could be a "fast_mode = yes/no" in the configuration, or do you have better name for it? ("keep_radiolink_open = yes/no", or something ) ljubex wrote And regarding cmgs, take a look into this manual for ZTE modem family:
http://download.maritex.com.pl/pdfs/wi/MG2636at.pdf
Thanks. I'll try to produce a custom version of smsd for you, so you can test this. In the configuration, what would be the name of this setting, any opinions? ljubex wrote I have several different types of modems with me, when I perform tests with ">" on all of them I'll come up with information.
Great  ljubex wrote Meanwhile ... is there a way to store a bunch of pdu's on modem (in order to save time), and later to trigger sending, message by message? I mean not by cmgw but with some burst-like command?
I do not know, but I think that there is no way. ljubex wrote Why it takes ~200ms between these two log points?
2011-03-10 11:03:54.472,6, GSM1: Sending SMS from to 38163440734
2011-03-10 11:03:54.672,7, GSM1: -> AT+CMMS=2;+CMGS=26
It's from point where smsd says "I'm about to send command" to "I've sent command", right?
Before any command is sent to the modem, smsd tries to read if a modem has something to say (like RING). This takes about 200 ms. A modem setting detect_unexpected_input = no disables this. |
|
|
#24 Thu Mar 10, 2011 13:02, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
This really helped !! I mean: detect_unexpected_input = no Performances are now almost perfect, the ony remaining difference is waiting for prompt character. Anyway, here is my current config: devices = GSM1,GSM2
outgoing = /var/spool/sms/outgoing
checked = /var/spool/sms/checked
incoming = /var/spool/sms/incoming
logfile = /var/log/smstools/smsd.log
infofile = /var/run/smstools/smsd.working
pidfile = /var/run/smstools/smsd.pid
failed = /var/spool/sms/failed
#sent = /var/spool/sms/sent
stats = /var/log/smstools/smsd_stats
loglevel = 7
logtime_ms = 1
keep_filename = no
receive_before_send = no
autosplit = 3
max_continuous_sending = 7200
[queues]
GSM1 = /var/spool/sms/GSM1
GSM2 = /var/spool/sms/GSM2
[GSM1]
device = /dev/ttyUSB8
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
report = no
sending_disabled = no
detect_unexpected_input = no
queues = GSM1
[GSM2]
device = /dev/ttyUSB18
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
report = no
sending_disabled = no
detect_unexpected_input = no
queues = GSM2
'smsdconf' Syntax Highlight powered by GeSHi Regarding the names, fast_mode is pretty much self explaining And prompt waiting ... maybe it's not necessary at all if it causes more trouble than benefit. I'll thest several cases by script and I'll show the results |
|
|
#25 Thu Mar 10, 2011 14:20, 159 months ago.
|
|
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
|
Here are some results. With my current modem (ZTE, K3565-Z, vendor:product = 19d2:0063) single-step at+cmgs does NOT work. However, when I put small delay between command string and pdu string supply (without waiting for prompt) it actually works (delay is aproximately 1-10ms). So, maybe it would be a good idea to put this delay as a parameter (in milliseconds), to be used on end user's own risk? |
Login and Post Reply
Page: 1 2 Next
Time in this board is UTC.
|
|
|
 |
|
|
 |
|
