Serial Data Parsing Question on Projector video wall
JohnMichnr
Junior Member
Hey All - I'm looking for help on a video wall that I am programming. I have 24 cube wall that I want to parse status on during power on & power off. I have a timeline sending out the status command to each wall for the duration of the poweron/off cycle ( I shut the status command off for the cubes that are either full on or full off) Then once a wall reports full on I ask for lamp 1 hours, the responce to that triggers the request for lamp 2 hours. At first everything seemed like it was working.... (like starting a novel with It was a dark and stormey night...)
But - I my program started locking up and the system was throwing out diagnostics error messages GetString - Error1 and CopyString Error-1. As I was looking into that I noticed that the wall would sometimes stop responding for 4-5 seconds - then send out a tons of responces, and some garbage thrown in there just for fun. (It responded with a power responce for a cube that was not there)
Now the problem that I am having is that the responce data has a start byte, no end byte, and is differing lengths (either 7 or 9 bytes long) I have to make sure that I have full and correct responce byte before hitting my parsing. And the start byte is $02 that can be used anywhere in the command - not just the start byte.
Here is a typical responce: $02,$04,$A0,$01,$00,$00,CS,$01
First $02 - Start Byte
$04 - data length
$A0 - Command (this case status)
$01,$00,$00 - three bytes of data (plust the command above equals the $04 for data length)
CS - check sum two's complement of length command and data (previous post)
$01 - what projector is responding.
Here is what I am using:
This code work fine to parse teh data, but only when I have a full and complete string going into it.
I need some help getting the data into the while loop. I think I need to create a loop that grabs each byte one at a time (get_buffer_char) then if the byte is a $02 - look 2 byte ahead and if it is a $A0 or $A1 (the only 2 I am working on) go back one byte to the data length and get that many bytes plus 2 (check sum and which projector)
But I am not that sure how to do it. Any suggestions? Any other thoughts?
Thanks in advance...
But - I my program started locking up and the system was throwing out diagnostics error messages GetString - Error1 and CopyString Error-1. As I was looking into that I noticed that the wall would sometimes stop responding for 4-5 seconds - then send out a tons of responces, and some garbage thrown in there just for fun. (It responded with a power responce for a cube that was not there)
Now the problem that I am having is that the responce data has a start byte, no end byte, and is differing lengths (either 7 or 9 bytes long) I have to make sure that I have full and correct responce byte before hitting my parsing. And the start byte is $02 that can be used anywhere in the command - not just the start byte.
Here is a typical responce: $02,$04,$A0,$01,$00,$00,CS,$01
First $02 - Start Byte
$04 - data length
$A0 - Command (this case status)
$01,$00,$00 - three bytes of data (plust the command above equals the $04 for data length)
CS - check sum two's complement of length command and data (previous post)
$01 - what projector is responding.
Here is what I am using:
string:
{
stack_var char sMsg[9]
stack_var integer nWhich
stack_var char nLamp
stack_var char nTempLamp[3]
stack_var integer nSize
stack_var char nCmd
stack_var integer N
stack_var integer nLocation
sWallRxbuf = "sWallRxbuf,data.text"
while(find_string(sWallRxbuf,"$02,$04,$A0",1) || find_string (sWallRxbuf,"$02,$05,$A1",1)
|| (LENGTH_STRING(sWallRxbuf) > 8))//lets parse this
{
sTrash = remove_string(sWallRxbuf,"$02",1)
nSize = get_buffer_char(sWallRxbuf)+2
if(length_string(sWallRxbuf)<nSize)
{
sWallRxbuf = "$02,nSize-2,sWallRxbuf" //put back what you just took out
}
else
{
sMsg = GET_BUFFER_STRING(sWallRxbuf,nSize)
nCmd = GET_BUFFER_CHAR(sMsg)
switch (nCmd)
{
case $A0: //status command
{
cBit[1] = sMsg[1] & 1
cbit[2] = sMsg[1] & 2
cbit[3] = sMsg[1] & 4
cbit[4] = sMsg[1] & 8
cbit[5] = sMsg[1] & 16
cbit[6] = sMsg[1] & 32
cbit[7] = sMsg[2] & 1
cbit[8] = sMsg[2] & 2
cbit[9] = sMsg[2] & 4
cbit[10] = sMsg[2] & 8
cbit[11] = sMsg[2] & 16
cbit[12] = sMsg[2] & 32
cbit[13] = sMsg[2] & 64
cbit[14] = sMsg[3] & 1
cbit[15] = sMsg[3] & 2
cbit[16] = sMsg[3] & 16
cbit[17] = sMsg[3] & 32
cbit[18] = sMsg[3] & 128
sTrash = get_buffer_string(sMsg,4)
nWhich = get_buffer_char(sMsg)
//cCubeRespCounter[nWhich]=0
for(N=1;N<=6;n++)
{
if(cbit[n]>0)
cCubeStatus[nWhich] = N-1
}
(*if(cLastCubeStatus[nWhich]<>cCubeStatus[nWhich])
{
call 'CUBE STATUS' (nWhich)
cLastCubeStatus[nWhich]=cCubeStatus[nWhich]
}*)
switch (cCubeStatus[nWhich])
{
case 4: //ERROR ON CUBE
{
cCubeError[nWhich] = ''
for(N=7;n<=18;n++)
{
cCubeError[nWhich] = "cCubeError[nWhich],';',nCubeErrorTxt[n]"
}
}
case 0: //cube is off
{
off[cCubeCheck[nWhich]]
off[cCooling[nWhich]]
}
case 5: //cube is on
{
off[cWarming[nWhich]]
off[cCubeCheck[nWhich]]
call 'WALL XMIT' (nGetLamp1,nWhich)
}
default:
{
on[cCubeCheck[nWhich]]
}
}
}
case $A1: //lamp hours status
{
nLamp = get_buffer_char(sMsg)
nTempLamp = get_buffer_string(sMsg,3)
sTRASH = get_buffer_char(sMsg)
nWhich = GET_BUFFER_CHAR(sMsg)
cCubeRespCounter[nWhich]=0
switch (nLamp)
{
case $20: //lamp 1
{
cCubeLampHours[nWhich][1] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]
call 'WALL XMIT' (nGetLamp2,nWhich)
}
case $40: //Lamp 2
{
cCubeLampHours[nWhich][2] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]
}
}
}
}
}
}
}
This code work fine to parse teh data, but only when I have a full and complete string going into it.
I need some help getting the data into the while loop. I think I need to create a loop that grabs each byte one at a time (get_buffer_char) then if the byte is a $02 - look 2 byte ahead and if it is a $A0 or $A1 (the only 2 I am working on) go back one byte to the data length and get that many bytes plus 2 (check sum and which projector)
But I am not that sure how to do it. Any suggestions? Any other thoughts?
Thanks in advance...
Comments
-
Here is my pseudocode of how I would do it:
If $02 is present
and next (data length) byte is present
{
Throw away everything before the $02
if at least the correct amount of data ($02 + 1 + data length + 2) is present
{
Cut the string out of the incoming stream leaving the rest for later
if the checksum is correct (call a separate subroutine)
and the projector number is sensible
{
Parse and interpret the comand
}
}
} -
I hate dealing with serial packets with no distinct terminaton, and especially with variable lengths. But, they are out there, and have to be dealt with.
I buffer the entire response, and peel off char by char. Generally, right after the start char is some indication of how big the packet is going to be: either a response code that has a specific data length to follow, a byte telling you how much data is to follow, or a combination of those. With that, you can build up your response in a holding variable, and finish it up when you have all the data you are expecting. If you get a new start character before the packet finished, throw it away (and perhaps send your debug window a message in case there is something wrong and not just junk data).
My experience is that equipment that uses this kind of protocol tends to be working at a fairly low level on the serial port; it's not bufered or interpreted within the device, and therefore more prone to errors on the output. So you have to be more careful with your error handling and make sure your code will deal with it gracefully rather than barfing all over your design. -
Yeah Great - you got any code samples that I could look at for teh character by character buffer? I figure I need a while statement in the data parsing somewhere - incase there is more than one command in the buffer (or a timeline just running to parse the data) not sure which way to go.
-
Basically, it's a matter of taking the code out of your STRING handler in the data event. Make a buffer of suitable size to hold everything that might come in at once, and put a CREAT_BUFFER statement in your DEFINE_START section. The master itself will now take every bit of feedback on that port, and append it to that buffer variable. Lets say you defined it as CHAR sVidWallBuffer[1024]. In DEFINE_START, put CREATE_BUFFER dvVidWall, sVIdWallBuffer. Then in DEFINE_PROGRAM, put a line that says something like IF(LENGHT_STRING(sVidWallBuffer)) doParseVidWall (or whatever you wish to name your parsing routine).
Your doParseVidWall function needs to have a local variable of the proper size for a single packet. What it needs to do is take the highest order character off the buffer, and append it to this local variable. For triggering purposes, I would also make a single char for holding each chunk so you can test it and decide what to do. You could do something like:DEFINE_FUNCTION doParseVidWall() { STACK_VAR CHAR cDigit[1] LOCAL_VAR CHAR sPacket[256] cDigit = LEFT_STRING(sVidWallBuffer, 1) // get first character sVidWallBuffer = RIGHT_STRING(sVidWallBuffer, LENGTH_STRING(sVidWallBuffer) - 1) // remove character from buffer IF(cDigit == "$02) CLEAR_BUFFER sPacket // we have a start char, clear data and start over ELSE { sPacket = "sPacket, cDigit" // append new data to the packet variable IF(LENGTH_STRING(sPacket) == <amount of data we are expecting> <code to process packet> } }
I'm sure there are other ways, that's just how I've handled similar situations. It would seem to me though, that the bulk of your problem is the STRING handler. What determines when that will fire on a NetLinx is when the master sees a pause in the data stream, it assumes the data is finished. So you are very likely to get incomplete data bursts using it if anything at all, including normal serial port delays in the equipment, makes that data pause. It will be in the next burst, it's not lost, but that is likely to throw off your parsing. Buffering it and testing char by char eliminates this. In my opinion, the STRING handler on a port is only suitable when you know for certain you will always get a full and complete packet of data every time - which is almost never with a serial port. I personally always buffer serial ports, and only use that handler with intra-master virtual messages. -
John,
This is the approach I?ve used in the past. It?s similar to what NMarkRoberts eluded to earlier. Here is some code:DEFINE_DEVICE dvProj = 5001:1:0 DEFINE_CONSTANT INTEGER nMinLength = 7 INTEGER nCmdPad = 3 //the $02 + checksum byte,+ proj# byte DEFINE_VARIABLE CHAR cBuffer[1024] DEFINE_FUNCTION fnParseBuffer () { CHAR junk[64] INTEGER junksize INTEGER cmdsize CHAR cmd[64] //do we at least have the minimum requirements for a valid message? IF (LENGTH_ARRAY(cBuffer)>=nMinLength && FIND_STRING(cBuffer,"$02",1)) { //lets check to see if any garbage is in fromt junksize = FIND_STRING(cBuffer,"$02",1) - 1 IF (junksize) { //strip the garbage out junk = GET_BUFFER_STRING(cBuffer,junksize) SEND_STRING 0, "'**** JUNK - ',ITOA(junksize),'junk characters getting tossed - ',junk" //lets spin out if the junk reducted our length to less than minimum IF (LENGTH_ARRAY(cBuffer) < nMinLength) RETURN } //we're still here so lets find out how long this command is cmdsize = cBuffer[2] + nCmdPad IF (LENGTH_ARRAY(cBuffer)>=cmdsize) { //do we have the entire command here? //we do so lets strip it cmd = GET_BUFFER_STRING(cBuffer,cmdsize) IF (fnCheckChecksum(cmd)) { //if checksum is valid fnProcessCommand(cmd) //pass this command on to be processed. } ELSE { SEND_STRING 0, "'Invalid checksum received with cmd: ',cmd" } } ELSE { SEND_STRING 0, "'Don''t have entire response yet'" //time to spin out and try again next go around RETURN } } } DEFINE_FUNCTION INTEGER fnCheckChecksum (char incoming[]) { //do whatever to check the checksum RETURN 1 //if checksum is good } DEFINE_FUNCTION fnProcessCommand (char cmd[]) { //see if it's a command you want to do something with and do it } DEFINE_START CREATE_BUFFER dvProj,cBuffer DEFINE_EVENT DEFINE_PROGRAM //I usually have this in mainline. //It's probably more correct to put it in a TIMELINE //Even more correcter (huh?) I think would be to put //this in the STRING: handler for dvProj but feel I //might miss something if I do. I'm curious where others put it fnParseBuffer() (***************THE END****************)
HTH -
Thanks for the more complete example Joe, I just can't get my brain going fast enough first thing in the morning to post full code blocks like that
... and the rest of the day, I'm posting bewtween compiles and reloads if I'm on at all.
But in answer to your question in the commented out portions: I would put it in mainline too. For all the reasons mentioned above, I don't think the STRING section is quite reliable for every possible situation where it might get only partial data. Perhaps it's just paranoia. -
Wow - Dudes... Thanks for the responce - I'm at a site with 4 of the cubes staring me in the face right now, and I'll try these to see what happens. I kind of like using the string handler for this stuff, although Dave I understand the reasons to try it elsewhere.
Again Thanks!
Joe I'll name a comemorative sub routine for you in my program... -
Can you make it a commemorative function instead?JohnMichnr wrote:Joe I'll name a comemorative sub routine for you in my program...
Let us know how the story ends. -
Can you make it a commemorative function instead?
I knew you were going to say that...
Actually I spent 6 quality hours with a 4 cube version on the wall yesterday with mixed results. The code that Joe sent out (thanks again) worked mostly, a few mode here and there and it seemed to come together. The big problem is that the wall goes stupid 5 seconds in and then waits 5 seconds and spits out a ton of stuff, some of it good, some of it not so good. The other thing the wall does is put out an ACK responce to some commands, any command going to cube #2, of $10,$02 like the same $02 that is supposed to be a start code for the responces... So the buffer gets one character off because it has 2 $02 in a row. But I can remove that just by looking for that ACK responce and deleting it before it gets to the buffer
The real issue now is that the Hebert Code (TM) and my parsing code is throwing out a bunch of GetString error 1 and CopyString Error 1 responces in diagnositcs. I don't know if that is cuaseing the other big error...
The customer did not want to purchase a touchpanel so we are using a G3 web panel for the control. That panel keeps locking up when I fire up the video wall as a whole. I can fire up individual cubes using another page and not have it lock up, but firing up the wall as a whole locks up the control panel, Sometimes. The only difference on the whole wall firing is that I add 2 more timelines going for a countdown timer on the webpanel and a general timer for the startup sequence of the system. So I don't know if that is the problem, or what. I don't have anymore time infront of the wall until I go onsite next week, but I programmed another cahssis to be a video wall emulator (except for the 5 second stupid period - but I am considering adding that) so I will be testing with my emulator until then.
The funny thing with this is - Ive been working so hard to get teh status to come up on a touchpanel page - I never really wondered if I really needed it. The wall consistan'y turns on & off with teh commands, I have never had that fail. Just problems with the status parse. So I am thinking of backing down my requirement a little. -
If you?re not already, compile with Debug Info and you will get a line numbers with the runtime errors which should be helpful in tracking down the culprit.John wrote:code is throwing out a bunch of GetString error 1 and CopyString Error 1 responces in diagnositcs.
There shouldn?t be any cases where the code I posted (unmodified) will cause any runtime errors.
Been there done that.John wrote:The funny thing with this is - Ive been working so hard to get teh status to come up on a touchpanel page - I never really wondered if I really needed it.
-
Well - I have just finished purging the Get String/Copy String errors from my program. I had help from Tech Support but it amounted to the following lines.
By the way - the debug info did not show any line numbers for these issues - don't know why
From Joes original code above ( And I did it as well a bunch of times)DEFINE_CONSTANT nCmdPad = 3 DEFINE_VARIABLE CHAR cBuffer[1024] DEFINE_FUNCTION fnParseBuffer () { CHAR junk[64] INTEGER junksize INTEGER cmdsize CHAR cmd[64] cmdsize = cBuffer[2] + nCmdPad
cmdsize is an integer, and 16 bits, 1 byte of cBuffer is a char and 8 bits. Relating an 8 bit byte to the 16 bit integer would through up those errors - as below
from the Diagnostic window.Line 25 :: GetString - Error 1 Tk=0x0001 - 16:01:34
Line 26 :: CopyString (Reference) - Error 1 S=0x0000 D=0x1011 - 16:01:34
tech support wanted me to BAND the single 8 bit byte with $FF before equating it to the integer. as belowcmdsize = (cBuffer[2] & $FF) + nCmdPad
going through and removing all of those in Joe's code and my own code got rid of all the errors.
The part I don't get is that the first code worked - even though it through out the error, and I know that I have done that routine before (equating a single portion of a character array to an integer variable) like everytime I parse Projector lamp hours from various projectors. And I have never seen the GetString/Copystring errors before. What bothers me is maybe I didn't check for them because I was getting the results I wanted. I went through the rest of the program and found the same error in the projector lamp parsing, and in 2 different devices check sum calculation.
And as it ends up my touch panel lock up was not caused by the errors - but by a run away while loop.
Any thought on this? -
I've seen a number of mysterious GetString errors, and they were generally similar to yours - the string passed to the function was not in a form the function was expecting. There are no line numbers in the debug output because it is a NetLinx library function, and you don't have the source for them, much less a debug-info compiled version or even the original source. The error is internal to the function. In those cases, it would be very helpful if the run time debugging was able to tell you what line of code called the function that generated the error.
-
DHawthorne wrote:In those cases, it would be very helpful if the run time debugging was able to tell you what line of code called the function that generated the error.
yes it would be... I ended up putting a bunch of send string 0 through out various areas to track down where in the code the errors popped up.
Categories
- All Categories
- 2.5K AMX General Discussion
- 922 AMX Technical Discussion
- 514 AMX Hardware
- 502 AMX Control Products
- 3 AMX Video Distribution Products
- 9 AMX Networked AV (SVSI) Products
- AMX Workspace & Collaboration Products
- 3.4K AMX Software
- 151 AMX Resource Management Suite Software
- 386 AMX Design Tools
- 2.4K NetLinx Studio
- 135 Duet/Cafe Duet
- 248 NetLinx Modules & Duet Modules
- 57 AMX RPM Forum
- 228 MODPEDIA - The Public Repository of Modules for Everyone
- 943 AMX Specialty Forums
- 2.6K AMXForums Archive
- 2.6K AMXForums Archive Threads
- 1.5K AMX Hardware
- 432 AMX Applications and Solutions
- 249 Residential Forum
- 182 Tips and Tricks
- 146 AMX Website/Forums