Operating system name and version: Windows XP SP3 and Cygwin
Version of smsd: v3.1.14-VVV-3 (special version prepared by keke)
Smsd installed from: sources
Name and model of a modem / phone: Siemens MC35
Interface: serial

Hello keke,
we have 10 GSM modems in our setup. Since your last help, system works fine. We have no more troubles with file movings and lockings.

Several days ago I noticed new issue with one of modems. Modem is called M9 and is connected to COM9.
Below are log fragments ( smsd and trouble ) related to that modem process:

2011-06-15 12:29:47,2, smsd: Smsd v3.1.14-VVV-3 started. 2011-06-15 12:29:49,5, M9: Modem handler 8 has started. PID: 2728. 2011-06-15 12:29:49,5, M9: Using check_memory_method 1: CPMS is used. ... < LAST RECORD IN TROUBLE LOG for that modem> 2011-06-21 08:34:12,7, M9: Running alarmhandler: /usr/src/myscripts/centaur_alarm_20090205.sh ALARM 2011-06-21 08:34:12 3 M9 "The modem answer was not OK: +CMSERROR: 21 (Short message transfer rejected)" 2011-06-21 08:34:12,7, M9: Done: alarmhandler, execution time 0 sec., status: 0(0) 2011-06-21 08:34:12,4, M9: Sending SMS to ... failed, trying time 35 sec. Retries: 2. 2011-06-21 08:34:28,5, M9: Everything ok now. < /LAST RECORD IN TROUBLE LOG for that modem> ... 2011-06-21 14:08:32,5, M9: SMS received, From: ... ... 2011-06-21 14:11:50,5, M9: SMS sent, Message_id: 79, To: ..., sending time 6 sec. ... 2011-06-21 14:12:48,5, M9: SMS sent, Message_id: 80, To: ..., sending time 9 sec. ... 2011-06-22 09:57:53,2, smsd: Smsd mainprocess is awaiting the termination of all modem handlers. PID: 6088. 2011-06-22 09:57:53,2, smsd: Smsd mainprocess terminated. PID 6088, was started 11-06-15 12:29:49. 2011-06-22 09:57:53,2, M6: Modem handler 5 terminated. PID: 5172, was started 11-06-15 12:29:49. 2011-06-22 09:57:53,2, M8: Modem handler 7 terminated. PID: 3760, was started 11-06-15 12:29:49. 2011-06-22 09:57:58,2, M7: Modem handler 6 terminated. PID: 3296, was started 11-06-15 12:29:49. 2011-06-22 09:57:58,2, M10: Modem handler 9 terminated. PID: 5220, was started 11-06-15 12:29:49. 2011-06-22 09:57:58,2, M2: Modem handler 1 terminated. PID: 5256, was started 11-06-15 12:29:49. 2011-06-22 09:57:58,2, M1: Modem handler 0 terminated. PID: 5836, was started 11-06-15 12:29:49. 2011-06-22 09:57:59,2, M3: Modem handler 2 terminated. PID: 2932, was started 11-06-15 12:29:49. 2011-06-22 09:57:59,2, M5: Modem handler 4 terminated. PID: 3896, was started 11-06-15 12:29:49. 2011-06-22 09:57:59,2, M4: Modem handler 3 terminated. PID: 5428, was started 11-06-15 12:29:49. 2011-06-22 09:58:13,2, smsd: Smsd v3.1.14-VVV-3 started. ... 2011-06-22 09:58:27,5, M9: SMS sent, Message_id: 81, To: ..., sending time 12 sec.

As can be seen, last activity of that modem was in 2011-06-21 14:12:48,5. After that modem stopped sending and receiving anythinhg. But there are no any trouble records in any of log for that period. I am not sure whether modem process was in memory, but during smsd restart at 2011-06-22 09:57:53,2 , that process is not listed in stop list. After smsd restart modem started working properly.

My questions:
a) is it possible that during such "hang up" modem process continues to read spooler dir ?
b) Have you seen such issue in your practice ?
c) what are possible reasons of the issue ?
d) what could be done to prevent it / diagnose further ?

My conf file was unchanged since previous discussion, let me know if you need it again.

And help you very much for kindly support.

The problem looks similar than you had once two months ago (#post3275). Modem process was stopped, but it is not known why. I assume that it did not stop by itself, because in that case there should be lines about that in the log.

You could create a script which monitors smsd processes, and creates some log and alarm when one or more modem processes are missing when mainprocess is still running. It will not tell who killed process, but you will get a notification faster.

If this issue happens once more, first check the ps list before restarting smsd, to see if there are all modem processes listed.

When modem process is stopped, or "hanged up", it does not continue sending and receiving. If you have implemented statistics, you could also use the file /var/spool/sms/stats/status for monitoring. From that file latest state and time of activity of all processes can be seen.

Thank you for above reminder
So our first task is to detect whether modem process remians in memory or not.