Fuzzy Math?

In the below section of code I do frequency conversion for a direct entry keypad for an AVR module I'm working on (B&K Ref 50). I display the digits as they are entered and the code below is for the enter button. I convert the string to a double and I have tried float but long ago doing this same thing I reaslized an issue with usinf float that using a double fixed. Anyway the line commented out is the "FM" book method that comes with the receiver which works fine on a calculator bbut when I run the code it's always off by one station (.2). The same formula in reverse posted beneath works fine. The line that currently being used is just something that seems to work that I did a few years back.

Why does the commented formula work on a calculator and not through the master?

CASE 14://ENTER / DONE BUTTON
     {
     if([vdvTP_AVR_1,BnK_FB_CH_BAND])// on = FM
	  {
	  STACK_VAR DOUBLE nFreqTemp ;
	    
	  nFreqTemp = ATOF(cFreqStr) ;
	  if (nFreqTemp >= 87.5 && nFreqTemp <= 107.9)
	       {
	       //nFreqTemp = (((nFreqTemp - 87.5)/0.2) + 1); //book method
	       nFreqTemp = (((nFreqTemp - 87.3)/.2) + .000017) //my old version
	       SEND_COMMAND vdvTP_AVR_1,"'Q_HEAD_STRING:(0,S,P1=FF,6=',ITOHEX(nFreqTemp2),';)'" ;
	       SEND_COMMAND vdvTP_AVR_1,"'Q_TAIL_STRING:(0,G,P1=FF,6;)'" ;
	       }           
	  }
     else //off = AM
	  {
	  STACK_VAR INTEGER nFreqTemp ;
	    
	  nFreqTemp = ATOI(cFreqStr) ;
	  if (nFreqTemp > 520 && nFreqTemp <= 1670)
	       {
	       nFreqTemp = (((nFreqTemp - 520)/10) + 2) ;
	       SEND_COMMAND vdvTP_AVR_1,"'Q_HEAD_STRING:(0,S,P1=FF,5=',ITOHEX(nFreqTemp),';)'" ;
	       SEND_COMMAND vdvTP_AVR_1,"'Q_TAIL_STRING:(0,G,P1=FF,5;)'" ;
	       }
	  }
     cFreqStr = '' ;
     fnSendVT(nTPIndx,BnK_VT_AMFM_FREQ,'1','Tuning....') ;//tp index,vt address#,states,text
     }

If my calculator comes up with HEX 3B for the freq I type in and is diplayed on the TP when I hit enter in diagnostics I see it send 3A and they the response form the device will be 3A and then my display will correctly display the frequency using the second conversion for 3A. Some how the formula come up with 3A in the master while the calculator show it should be 3B and the subsquent display is of by .2 from what I typed in.

I'm quite confused and running out of things to try. Although I knew the old formula worked I didn't want to give in and use it but after a couple of hours I couldn't take it anymore.

From the B&K manual:

Note 6: 200 kHz FM step tuning (USA)
if value = 0,
indicates uninitialized ?OPEN? frequency
else,
(((value - 1)* 0.20) + 87.5) = frequency in MHz
((frequency ? 87.5) / 0.20) + 1 = value
For example of FM Frequency of 2Eh:
(((2Eh - 1) * 0.20) + 87.5 = 96.5 MHz
((96.5 MHz ? 87.5) / 0.20) + 1 = 29h
NOTE: If the recalled preset contains an OPEN frequency

Example of strings and freaks:

98.7 MHz - (0,S,P1=FF,6=39;)
98.9 MHz - (0,S,P1=FF,6=3A;)
99.1 MHz - (0,S,P1=FF,6=3B;)
99.3 MHz - (0,S,P1=FF,6=3C;)
99.5 MHz - (0,S,P1=FF,6=3D;)
99.7 MHz - (0,S,P1=FF,6=3E;)
99.9 MHz - (0,S,P1=FF,6=3F;)