SMS Server Tools 3
This site is hosted by Kekekasvi.com
 Menu
Basic information:
Additional information:
Support:
Get SMS Server Tools 3:
Additional Options

 Sponsored links

 Search
Custom Search

 Visitor locations
 
 SMS Server Tools 3 Community
Welcome, Guest. Please login or register. Tue Apr 13, 2021 14:32
SMSTools3 Community » Help and support Bottom

fast sms + queues

Login and Post Reply

Page:  1  2  Next

Author Post
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Operating system name and version: Debian GNU/Linux squeeze/sid (2.6.32-3-686)
Version of smsd: 3.1.11
Smsd installed from: package repository
Name and model of a modem / phone: ZTE K3565-Z
Interface: USB


Hi everybody,
I'm trying to send a bunch of SMS over several usbserial modems. I'm trying to send all sms in fast sms sending mode so I've put at+cmss=2 as init string. When I use one modem at a time and use global spool (/var/spool/sms/outgoing) everything goes well and it takes aproximately 0.8 - 0.9s per SMS, which is ok. But when I try to send over two modems where I have a queue for each of them - the speed goes normal (not fast) and it takes about 2.5s per SMS (which is intolerably slow in my case). When I start two separated smsd instances, each having separate config and pid file, then it goes fast again. So is there a chance to make sending over queues to go in fast speed mode?

Here is my global config file:
devices = GSM1,GSM2
outgoing = /var/spool/sms/outgoing
checked = /var/spool/sms/checked
incoming = /var/spool/sms/incoming
logfile = /var/log/smstools/smsd.log
infofile = /var/run/smstools/smsd.working
pidfile = /var/run/smstools/smsd.pid
failed = /var/spool/sms/failed
sent = /var/spool/sms/sent
stats = /var/log/smstools/smsd_stats
receive_before_send = no
autosplit = 3
max_continuous_sending = 7200

[queues]
GSM1 = /var/spool/sms/GSM1
GSM2 = /var/spool/sms/GSM2

[GSM1]
device = /dev/ttyUSB3
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
rtscts = yes
cs_convert = yes
report = no
sending_disabled = no
queues = GSM1

[GSM2]
device = /dev/ttyUSB8
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
rtscts = yes
cs_convert = yes
report = no
sending_disabled = no
queues = GSM2
 
 
'smsdconf' Syntax Highlight powered by GeSHi


Also, is there a chance to force multiple modems to serve one queue? (hundreds of modems).

Any help will be appreciated :)

Best regards,
Ljubomir

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
First, you need to upgrade to 3.1.12 or later. Debian has migrated 3.1.14-1 to testing today, so it's available from the repository.

In the 3.1.11 there is a bug when more than one modems are serving the same queue. Lock files are not checked by default, and this may cause slow performance in some cases. A modem setting trust_spool = no helps for this (with 3.1.11), but there is then a one second delay when searching for a file. This probably is too much for you.

By default all modems will serve the same queue, which is a "checked" directory, when queues are not defined.

You have had headers "Queue: GSM1" or "Queue: GSM2" in SMS files, because without this definition all messages are targeted to GSM1. It may be possible that other modem had nothing to send for few seconds, because some number of messages were spooled to another modem. This causes that the radio link is closed (typically after 3 seconds of idle time), and benefit of AT+CMMS is lost at that time.

It could be useful to see some pieces of a log, but first upgrade to 3.1.14 which provides more infomation about spooling.

Some settings probably can be used to speed up the spooling, but before I recommend anything, I need to know how do you exactly create outgoing SMS files, and what is the number of files in outgoing directory as a peak?

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Hi Keke, thanx for fast reply !

I guess that problem is actually more profound than I thought. The speed is always normal, never really gets to fast mode: I only measured the speed of emptying outgoing directory. So, didn't realize that all messages are moved to checked or some other queue and than being processed.

Anyway, I upgraded to 3.1.14. The speed I'm trying to get is definitely achieved with this modem by hand: I've made simple script that just triggers at command for sms send, waits for OK and so on ... Am I wrong or this starts to look like feature request? Simply what has to be done is to invoke at+cmss=2 seconds before sending is started. I put keep_open = no (hoping that init string will be sent to modem in apropriate time, but this didn't help as well).

Here is a log of following configuration:
- Two modems
- Two separate queues, each modem serving one queue
- Copied 3 messages in each queue at (almost) same time



Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
If you are sure that "More Messages to Send" works in your GSM network, we could patch the code. In this case you should download the latest package and prepare to compile it. You will also need development version of libmm. If you can do this, I will show later how to change the code.

It is known that CMMS is not supported by many modems, and also it's not supported by many networks. But for those users who have the support, this feature might be very valuable. It looks like I'm going to include a modem setting "fast_mode" in the next version. ;) But let's test it first...

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Deal ! :) I'm waiting for further instructions, now going to grab the source code ....

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
Okay. Edit the file smsd.c, search the following and change the highlighted line:

  // Compose the modem command
  make_pdu(to,text,textlen,alphabet,flash,report,with_udh,udh_data,DEVICE.mode,pdu,validity, replace_msg, system_msg, to_type, smsc);
  if (strcasecmp(DEVICE.mode,"old")==0)
    sprintf(command,"AT+CMGS=%i\r",(int)strlen(pdu)/2);
  else
    sprintf(command,"AT%s+CMMS=2;+CMGS=%i\r", (DEVICE.verify_pdu)? "E1" : "", (int)strlen(pdu)/2-1); // 3.1.4: verify_pdu
 
'c' Syntax Highlight powered by GeSHi


Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Hmmm, still same performances :( Is it possible that spool engine takes to much time to feed the modem? (didn't read the source actually, just coming up with the idea ....)

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
From the log we should see how much the time is spent between each SMS.

About your testing script, does it generate timestamps which will show the difference?

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
Another change, it may help but I'm not sure. File smsd.c:

  else
  {
    // 3.1:
    if (0) // if (DEVICE.sending_disabled == 0 && DEVICE.check_network)
    {
      switch (wait_network_registration(1, 100))
      {
        case -1:
          STATISTICS->usage_s+=time(0)-start_time;
          flush_smart_logging();
          return 1;

        case -2:
          STATISTICS->usage_s+=time(0)-start_time;
          flush_smart_logging();
          return 3;
      }
    }
  }

  // Compose the modem command
 
'c' Syntax Highlight powered by GeSHi


Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Hi, here are logs and comparation.

First, here is log of pathced smsd:



Now, here is a script I'm testing cmms with:



So as you can see the message is stored into memory and it is simply resent every time. I tried with text and pdu message, the results are completely the same (regarding the speed). Here are timestamps:



The script is quite unstable and unuseful, I just made it for test as you can see :)

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Ah I now saw new reply from you :) When I said "patched" I meant on first one, not second one .... Be online back in a few hours and then I'll let you know if second one helped ;)

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
While waiting for the new log...

We are now talking about two completely different features. In your first post you were talking about AT+CMSS, but in the configuration AT+CMMS was used. I was thinking that the CMSS is a typo, as this feature is not supported by smstools, and even if it is implemented, it does not belong to the init string.

CMMS may work, but it depends on the network, as said. I have one more patch available, but let's see the current result first.

CMSS will be problematic, if incoming messages are read. Stored messages should be in different memory, otherwise smsd will read them out and mark as "unsupported messages". To get this functionality work well, smsd should create stored message, send it from memory to multiple recipients, and then delete message. Lot of new functionality and testing will be needed. I think that if "lightning fast" speed is required for sending, external script like yours could be used.

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Hi again,

I would say that with both patches the results are quite the same (new log is included).

But reported time of sending is 1s, it actually fits performances provided by at+cmms. But it remains unresolved where smsd spends it's "spare time", because resulting time of sending is a little bit longer (2 seconds per SMS, maybe more).



Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
ljubex wrote
I would say that with both patches the results are quite the same (new log is included).

But reported time of sending is 1s, it actually fits performances provided by at+cmms. But it remains unresolved where smsd spends it's "spare time", because resulting time of sending is a little bit longer (2 seconds per SMS, maybe more).

Sending time is now better because network registration is not checked after the first message was sent.

With "at+cmms" you meant at+cmss, which is used in your script, right? Because those are two different things, results cannot be directly compared.

Make the following changes:

In the global part of smsd.conf, define logtime_us = yes. The next log will show better time resolution.

In the file extras.c, apply the following change:

  if (!found)
    *filename = 0;
  else
  {
    strcpy(filename, fname);

    // 3.1.12:
    i = (int)(time_usec() - start_time) / 100000;
    // if (i > 10)
      writelogfile((i >= 50)? LOG_NOTICE : LOG_DEBUG, 0, "Took %.1f seconds to get a file %s, lost %i times, %i SMS files and %i LOCK files seen.", (double)i / 10, fname, lost_count, files_count, locked_count);
  }

#ifdef DEBUGMSG
  printf("## result for dir %s: %s\n\n", dir, filename);
#endif
  return found;
}
 
'c' Syntax Highlight powered by GeSHi


After those changes do the test again and show the log.

Then, remove the patch shown in post #6 (file smsd.c):

  // Compose the modem command
  make_pdu(to,text,textlen,alphabet,flash,report,with_udh,udh_data,DEVICE.mode,pdu,validity, replace_msg, system_msg, to_type, smsc);
  if (strcasecmp(DEVICE.mode,"old")==0)
    sprintf(command,"AT+CMGS=%i\r",(int)strlen(pdu)/2);
  else
    sprintf(command,"AT%s+CMGS=%i\r", (DEVICE.verify_pdu)? "E1" : "", (int)strlen(pdu)/2-1); // 3.1.4: verify_pdu
 
'c' Syntax Highlight powered by GeSHi


Add a new patch in smsd.c:

  // 3.1beta7: Now logged only if a message file contained Report:yes.
  if (report == 1 && !DEVICE.incoming)
    writelogfile(LOG_NOTICE, 0, "Cannot receive status report because receiving is disabled");

  if ((quick==0 || (*smsc && !DEVICE.smsc_pdu)) && DEVICE.sending_disabled == 0)
  {
    int i;

    i = initialize_modem_sending(smsc);
    if (i)
    {
      STATISTICS->usage_s+=time(0)-start_time;

      flush_smart_logging();

      return (i == 7)? 3 : 1;
    }

    put_command("AT+CMMS=2\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);
  }
  else
 
'c' Syntax Highlight powered by GeSHi


This will show the execution time of AT+CMMS, and if it has remarkable effect when it was used together with AT+CMGS, every time when SMS was sent. Run test and show this log too.

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
I'm on my way ,,,, I've just enabled miliseconds log, few minutes ago :)

And regarding cmms/cmss:

Script is using at+cmss (send mesage from storage) because there was manualy stored pdu on the modem, so it simply invokes mesage sending from storage.
And at+cmms=2 (more messages to send) is invoked manually before sending. Also, when it is set to 2, there is no need to trigger it every time: only first message takes normal time, all of the following ones are going fast.

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
And here is the log according to latest instructions:



Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Okay, so as far as I can see, there are two main reasons for difference in speed:
- First is this 0.5s acquiring of next message from spool
- And second one is two step scenario: smsd is waiting for ">" before submitting pdp string: in my scenario message is transfered once and sending is triggered with no reason to wait for modem once again

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
In many at command references I saw that at+cmgs is documented as "single step" command, meaning that <CR> character and PDU are supplied together with at+cmgs. That would save 300ms in this particular case. So, 500ms of message preparation would be the last remaining difference in performances I guess.

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
ljubex wrote
And regarding cmms/cmss:

Script is using at+cmss (send mesage from storage) because there was manualy stored pdu on the modem, so it simply invokes mesage sending from storage.

Yes of course, but "at+cmss=2 as init string" sounds no good ;). It's not supported by smstools.

ljubex wrote
And at+cmms=2 (more messages to send) is invoked manually before sending. Also, when it is set to 2, there is no need to trigger it every time: only first message takes normal time, all of the following ones are going fast.

Yes, the manual says so. However, your log says that after +CMMS=2 was moved to the first time initialization, sending time was always 2 sec (~1500 ms). Before that it was reported as 1 sec.

I hope that you can do two more tests, now when logtime_ms is enabled:

First, change the
put_command("AT+CMMS=2\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);
to
put_command("AT+CMMS=0\r", answer ,sizeof(answer), 1, EXPECT_OK_ERROR);.

I would like to see what is the sending time when CMMS is disabled.

Second, remove that CMMS= command line (comment it out with //), and apply the patch in post #6 again. I would like to see what is the exact sending time when CMMS is enable during every sending.

Please do these two tests first, before changing anything else.


ljubex wrote
Okay, so as far as I can see, there are two main reasons for difference in speed:
- First is this 0.5s acquiring of next message from spool

If you have a modem setting trust_spool = no in the configuration, remove it. I recommended to use it with 3.1.11 only, because of a bug in that version.

ljubex wrote
- And second one is two step scenario: smsd is waiting for ">" before submitting pdp string: in my scenario message is transfered once and sending is triggered with no reason to wait for modem once again

ljubex wrote
In many at command references I saw that at+cmgs is documented as "single step" command, meaning that <CR> character and PDU are supplied together with at+cmgs. That would save 300ms in this particular case. So, 500ms of message preparation would be the last remaining difference in performances I guess.

Can you give a link to a manual which documents +CMGS like this? With a quick search and view I did not find any. All manuals said that it is necessary to wait the prompt. If some modem will work as "single step", it would be very interesting and likely I will include this feature in the next version. Hopefully your modem supports this, so I can use you to test the change to the code ;).

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Ah sorry, now I saw a typo in post :) It's ok in my smsd.conf ;) I'll have test results soon, and yes, there is a line:
trust_spool = no

so I'll test without it.

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
First scenario (cmms disabled completely):


Second scenario (cmms set every time):


The difference is pretty obvious even by naked eye :)

And regarding cmgs, take a look into this manual for ZTE modem family:
http://download.maritex.com.pl/pdfs/wi/MG2636at.pdf

Here, and on many other manuals I saw cmgs documented as:
AT+CMGS=<length><CR>
<pdu><Ctrl-Z/ESC>

So, only important thing is <CR> in a string. I believe that ">" is sent unsyncronously from modem side, and input buffer of a modem is not affected. But of course this looks like strongly modem-dependent issue. I have several different types of modems with me, when I perform tests with ">" on all of them I'll come up with information.

Meanwhile ... is there a way to store a bunch of pdu's on modem (in order to save time), and later to trigger sending, message by message? I mean not by cmgw but with some burst-like command?


« Last edit by ljubex on Thu Mar 10, 2011 10:27, 122 months ago. »
Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Why it takes ~200ms between these two log points?

2011-03-10 11:03:54.472,6, GSM1: Sending SMS from to 38163440734
2011-03-10 11:03:54.672,7, GSM1: -> AT+CMMS=2;+CMGS=26

It's from point where smsd says "I'm about to send command" to "I've sent command", right?

Administrator
Registered:
May 2009
Location: Jyväskylä, Finland
ljubex wrote
The difference is pretty obvious even by naked eye :)

Yes, CMMS works as expected and there is no reason to set it every time. I will do those changes, as well as changes to "check registration" to the code. This feature could be a "fast_mode = yes/no" in the configuration, or do you have better name for it? ("keep_radiolink_open = yes/no", or something ;) )

ljubex wrote
And regarding cmgs, take a look into this manual for ZTE modem family:
http://download.maritex.com.pl/pdfs/wi/MG2636at.pdf

Thanks. I'll try to produce a custom version of smsd for you, so you can test this. In the configuration, what would be the name of this setting, any opinions?

ljubex wrote
I have several different types of modems with me, when I perform tests with ">" on all of them I'll come up with information.

Great 8)

ljubex wrote
Meanwhile ... is there a way to store a bunch of pdu's on modem (in order to save time), and later to trigger sending, message by message? I mean not by cmgw but with some burst-like command?

I do not know, but I think that there is no way.

ljubex wrote
Why it takes ~200ms between these two log points?

2011-03-10 11:03:54.472,6, GSM1: Sending SMS from to 38163440734
2011-03-10 11:03:54.672,7, GSM1: -> AT+CMMS=2;+CMGS=26

It's from point where smsd says "I'm about to send command" to "I've sent command", right?

Before any command is sent to the modem, smsd tries to read if a modem has something to say (like RING). This takes about 200 ms. A modem setting detect_unexpected_input = no disables this.

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
This really helped !! :) I mean:
detect_unexpected_input = no

Performances are now almost perfect, the ony remaining difference is waiting for prompt character. Anyway, here is my current config:

devices = GSM1,GSM2
outgoing = /var/spool/sms/outgoing
checked = /var/spool/sms/checked
incoming = /var/spool/sms/incoming
logfile = /var/log/smstools/smsd.log
infofile = /var/run/smstools/smsd.working
pidfile = /var/run/smstools/smsd.pid
failed = /var/spool/sms/failed
#sent = /var/spool/sms/sent
stats = /var/log/smstools/smsd_stats
loglevel = 7
logtime_ms = 1
keep_filename = no
receive_before_send = no
autosplit = 3
max_continuous_sending = 7200

[queues]
GSM1 = /var/spool/sms/GSM1
GSM2 = /var/spool/sms/GSM2

[GSM1]
device = /dev/ttyUSB8
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
report = no
sending_disabled = no
detect_unexpected_input = no
queues = GSM1

[GSM2]
device = /dev/ttyUSB18
init = at+cmms=2
incoming = no
mode = new
baudrate = 115200
report = no
sending_disabled = no
detect_unexpected_input = no
queues = GSM2
 
 
'smsdconf' Syntax Highlight powered by GeSHi


Regarding the names, fast_mode is pretty much self explaining :) And prompt waiting ... maybe it's not necessary at all if it causes more trouble than benefit. I'll thest several cases by script and I'll show the results ;)

Member
Registered:
Mar 2011
Location: Belgrade, Serbia
Topic owner
Here are some results. With my current modem (ZTE, K3565-Z, vendor:product = 19d2:0063) single-step at+cmgs does NOT work. However, when I put small delay between command string and pdu string supply (without waiting for prompt) it actually works (delay is aproximately 1-10ms). So, maybe it would be a good idea to put this delay as a parameter (in milliseconds), to be used on end user's own risk? :)

Login and Post Reply

Page:  1  2  Next

SMSTools3 Community » Help and support Top

 
Time in this board is UTC.  

Privacy Policy   SMS Server Tools 3 Copyright © Keijo Kasvi.