Dropping the Highest Order Bit

John Svee at Data Conversion Resource / Trusted Data Solutions found one of our posts/pages at this site, and has provided us a GOLDMINE of knowledge, as to how to decipher the data that I've extracted from all of the tapes that I reference as part of this Microtech / Point 4 restoration project.

Here it is, in his own words:

To whom it may concern,

When doing research for a current project involving 4-track QIC tape data (DC-300 tapes) I came across “http://microtechm1.blogspot.com/2015/11/microtech-m134-iris-r7-tape-read.html”.

I downloaded and worked with the tape file (IRIS-Track 0Fthru2F-whole file.bin) and made some observations.

1.) Some data is readable ASCII as you pointed out.

00D58270 00 00 00 00 00 60 0D 0A 4D 41 47 20 54 41 50 45 .....`..MAG TAPE

00D58280 20 52 45 4C 49 41 42 49 4C 49 54 59 20 4F 50 30 RELIABILITY OP0

00D58290 32 30 30 30 38 2D 30 30 31 43 00 00 0D 0A 53 45 20008-001C....SE

00D582A0 4C 45 43 54 20 44 45 56 49 43 45 20 41 44 44 52 LECT DEVICE ADDR

00D582B0 45 53 53 3A 20 00 0D 0A 53 45 4C 45 43 54 20 55 ESS: ...SELECT U

00D582C0 4E 49 54 3F 20 00 0D 0A 54 59 50 45 20 28 31 29 NIT? ...TYPE (1)

00D582D0 20 4E 52 5A 49 20 28 32 29 20 50 45 3F 20 00 00 NRZI (2) PE? ..

00D582E0 0D 0A 53 54 41 54 55 53 20 44 45 4E 53 49 54 59 ..STATUS DENSITY

00D582F0 20 42 49 54 20 34 20 28 30 29 20 4F 46 46 2C 20 BIT 4 (0) OFF,

00D58300 28 31 29 20 4F 4E 20 3F 20 00 0D 0A 53 45 4C 45 (1) ON ? ...SELE

00D58310 43 54 20 54 45 53 54 0D 0A 20 28 31 29 20 52 45 CT TEST.. (1) RE

00D58320 4C 49 41 42 49 4C 49 54 59 20 54 45 53 54 0D 0A LIABILITY TEST..

00D58330 20 28 32 29 20 49 4E 54 45 52 43 48 41 4E 47 45 (2) INTERCHANGE

00D58340 20 54 45 53 54 28 52 45 41 44 20 26 20 57 52 49 TEST(READ & WRI

00D58350 54 45 29 0D 0A 20 28 33 29 20 49 4E 54 45 52 43 TE).. (3) INTERC

00D58360 48 41 4E 47 45 20 54 45 53 54 28 52 45 41 44 20 HANGE TEST(READ

00D58370 4F 4E 4C 59 29 0D 0A 20 28 34 29 20 54 45 53 54 ONLY).. (4) TEST

00D58380 20 4C 4F 4F 50 20 42 55 49 4C 44 45 52 0D 0A 20 LOOP BUILDER..

00D58390 20 00 0D 0A 2D 3F 2D 20 00 00 09 00 A0 D2 C5 41 ...-?- .... ÒÅA

00D583A0 44 A0 50 41 D2 C9 D4 59 AC 00 A0 D2 C5 C3 CF D2 D PAÒÉÔY¬. ÒÅÃÏÒ

00D583B0 44 A3 A0 00 BD C3 48 C5 C3 4B A0 D7 CF D2 44 00 D£ .½ÃHÅÃK ×ÏÒD.

00D583C0 BD 53 D4 41 D4 55 53 A0 41 C6 D4 C5 D2 A0 41 A0 ½SÔAÔUS AÆÔÅÒ A

00D583D0 D2 C5 41 44 A0 CF 4E A0 55 4E C9 D4 A0 00 A0 D7 ÒÅAD ÏN UNÉÔ . ×

00D583E0 D2 C9 D4 C5 A0 50 41 D2 C9 D4 59 AC A0 00 A0 42 ÒÉÔÅ PAÒÉÔY¬ . B

00D583F0 41 44 A0 D4 41 50 C5 AC A0 00 D7 D2 C9 D4 C5 A0 AD ÔAPŬ .×ÒÉÔÅ

00D58400 CC CF C3 4B A0 CF 4E A0 55 4E C9 D4 A0 00 BD 53 ÌÏÃK ÏN UNÉÔ .½S

00D58410 C5 4E D4 AC D2 C5 C3 56 44 2E A0 41 44 44 D2 C5 ÅNÔ¬ÒÅÃVD. ADDÒÅ

00D58420 53 53 A0 D2 C5 47 A0 C6 41 C9 CC A0 D4 CF A0 CC SS ÒÅG ÆAÉÌ ÔÏ Ì

00D58430 CF 41 44 2E A0 55 4E C9 D4 A0 00 00 BD C5 D2 D2 ÏAD. UNÉÔ ..½ÅÒÒ

00D58440 CF D2 A0 53 D4 41 D4 55 53 A0 41 C6 D4 C5 D2 A0 ÏÒ SÔAÔUS AÆÔÅÒ

00D58450 41 A0 22 C3 22 A0 50 55 CC 53 C5 2E A0 55 4E C9 A "Ã" PUÌSÅ. UNÉ

00D58460 D4 A0 00 00 A0 41 44 44 D2 C5 53 53 2B D7 C3 AC Ô .. ADDÒÅSS+×ì

00D58470 44 C9 42 A0 D2 C5 41 44 C9 4E 47 2E A0 D7 D2 C9 DÉB ÒÅADÉNG. ×ÒÉ

00D58480 D4 C5 A0 55 4E C9 D4 A0 00 00 BD 53 D4 41 D4 55 ÔÅ UNÉÔ ..½SÔAÔU

00D58490 53 A0 41 C6 D4 C5 D2 A0 41 A0 D7 D2 C9 D4 C5 2E S AÆÔÅÒ A ×ÒÉÔÅ.

00D584A0 A0 55 4E C9 D4 A0 00 00 BD 53 D4 41 D4 55 53 A0 UNÉÔ ..½SÔAÔUS

00D584B0 41 C6 D4 C5 D2 A0 41 A0 53 50 41 C3 C5 2E A0 55 AÆÔÅÒ A SPAÃÅ. U

00D584C0 4E C9 D4 A0 00 00 BD 53 D4 41 D4 55 53 A0 41 C6 NÉÔ ..½SÔAÔUS AÆ

00D584D0 D4 C5 D2 A0 41 A0 C5 D2 41 53 C5 2E A0 55 4E C9 ÔÅÒ A ÅÒASÅ. UNÉ

00D584E0 D4 A0 00 00 BD 53 D4 41 D4 55 53 A0 41 C6 D4 C5 Ô ..½SÔAÔUS AÆÔÅ

00D584F0 D2 A0 D7 D2 C9 D4 C5 A0 C5 CF C6 2E A0 55 4E C9 Ò ×ÒÉÔÅ ÅÏÆ. UNÉ

00D58500 D4 A0 00 00 4E CF A0 C5 CF C6 A0 53 D4 41 D4 55 Ô ..NÏ ÅÏÆ SÔAÔU

00D58510 53 A0 44 C5 D4 C5 C3 D4 C5 44 A0 CF 4E A0 D7 D2 S DÅÔÅÃÔÅD ÏN ×Ò

00D58520 C9 D4 C5 A0 C5 CF C6 2E A0 55 4E C9 D4 A0 00 00 ÉÔÅ ÅÏÆ. UNÉÔ ..

00D58530 8D 0A 41 44 44 D2 09 A0 A0 47 CF CF 44 09 A0 A0 .ADDÒ. GÏÏD.

00D58540 42 41 44 09 D7 CF D2 44 09 D2 C5 C3 A3 09 55 4E BAD.×ÏÒD.ÒÅã.UN

00D58550 C9 D4 00 00 8D 0A 4E CF A0 D2 C5 41 44 59 A0 D7 ÉÔ.. .NÏ ÒÅADY ×

00D58560 D2 C9 D4 C5 A0 C5 4E 41 42 CC C5 44 A0 55 4E C9 ÒÉÔÅ ÅNABÌÅD UNÉ

00D58570 D4 53 A0 41 56 41 C9 CC 00 00 D2 CF D4 41 D4 C5 ÔS AVAÉÌ..ÒÏÔAÔÅ

00D58580 A0 D4 41 50 C5 53 A0 C6 D2 CF 4D A0 55 4E C9 D4 ÔAPÅS ÆÒÏM UNÉÔ

00D58590 A0 CF A0 D4 CF A0 B1 AC A0 B1 A0 D4 CF A0 B2 AC Ï ÔÏ ±¬ ± ÔÏ ²¬

00D585A0 A0 C5 D4 C3 A0 50 D2 C5 53 53 A0 D4 C5 CC C5 D4 ÅÔÃ PÒÅSS ÔÅÌÅÔ

00D585B0 59 50 C5 A0 4B C5 59 A0 D4 CF A0 C3 CF 4E D4 C9 YPÅ KÅY ÔÏ ÃÏNÔÉ

00D585C0 4E 55 C5 2E 00 00 55 4E C9 D4 09 A0 09 00 D7 D2 NUÅ...UNÉÔ. ..×Ò

00D585D0 C9 D4 C5 A0 50 41 D2 C9 D4 59 09 00 D2 C5 41 44 ÉÔÅ PAÒÉÔY..ÒÅAD

00D585E0 A0 A0 50 41 D2 C9 D4 59 09 00 D7 D2 C9 D4 C5 A0 PAÒÉÔY..×ÒÉÔÅ

00D585F0 C5 D2 D2 CF D2 53 09 00 D2 C5 41 44 A0 A0 C5 D2 ÅÒÒÏÒS..ÒÅAD ÅÒ

2.) Some data is readable when you drop the high order bit (Like WordStar type data). See data beginning with “READ PARITY”.

Dropping this bit does not affect data with ASCII values below 128. I imagine any binary portions of this file could potentially be meant to contain > 127 values.

00D58270  00 00 00 00 00 60 0D 0A 4D 41 47 20 54 41 50 45    .....`..MAG TAPE

00D58280  20 52 45 4C 49 41 42 49 4C 49 54 59 20 4F 50 30     RELIABILITY OP0

00D58290  32 30 30 30 38 2D 30 30 31 43 00 00 0D 0A 53 45    20008-001C....SE

00D582A0  4C 45 43 54 20 44 45 56 49 43 45 20 41 44 44 52    LECT DEVICE ADDR

00D582B0  45 53 53 3A 20 00 0D 0A 53 45 4C 45 43 54 20 55    ESS: ...SELECT U

00D582C0  4E 49 54 3F 20 00 0D 0A 54 59 50 45 20 28 31 29    NIT? ...TYPE (1)

00D582D0  20 4E 52 5A 49 20 28 32 29 20 50 45 3F 20 00 00     NRZI (2) PE? ..

00D582E0  0D 0A 53 54 41 54 55 53 20 44 45 4E 53 49 54 59    ..STATUS DENSITY

00D582F0  20 42 49 54 20 34 20 28 30 29 20 4F 46 46 2C 20     BIT 4 (0) OFF,

00D58300  28 31 29 20 4F 4E 20 3F 20 00 0D 0A 53 45 4C 45    (1) ON ? ...SELE

00D58310  43 54 20 54 45 53 54 0D 0A 20 28 31 29 20 52 45    CT TEST.. (1) RE

00D58320  4C 49 41 42 49 4C 49 54 59 20 54 45 53 54 0D 0A    LIABILITY TEST..

00D58330  20 28 32 29 20 49 4E 54 45 52 43 48 41 4E 47 45     (2) INTERCHANGE

00D58340  20 54 45 53 54 28 52 45 41 44 20 26 20 57 52 49     TEST(READ & WRI

00D58350  54 45 29 0D 0A 20 28 33 29 20 49 4E 54 45 52 43    TE).. (3) INTERC

00D58360  48 41 4E 47 45 20 54 45 53 54 28 52 45 41 44 20    HANGE TEST(READ

00D58370  4F 4E 4C 59 29 0D 0A 20 28 34 29 20 54 45 53 54    ONLY).. (4) TEST

00D58380  20 4C 4F 4F 50 20 42 55 49 4C 44 45 52 0D 0A 20     LOOP BUILDER..

00D58390  20 00 0D 0A 2D 3F 2D 20 00 00 09 00 20 52 45 41     ...-?- .... REA

00D583A0  44 20 50 41 52 49 54 59 2C 00 20 52 45 43 4F 52    D PARITY,. RECOR

00D583B0  44 23 20 00 3D 43 48 45 43 4B 20 57 4F 52 44 00    D# .=CHECK WORD.

00D583C0  3D 53 54 41 54 55 53 20 41 46 54 45 52 20 41 20    =STATUS AFTER A

00D583D0  52 45 41 44 20 4F 4E 20 55 4E 49 54 20 00 20 57    READ ON UNIT . W

00D583E0  52 49 54 45 20 50 41 52 49 54 59 2C 20 00 20 42    RITE PARITY, . B

00D583F0  41 44 20 54 41 50 45 2C 20 00 57 52 49 54 45 20    AD TAPE, .WRITE

00D58400  4C 4F 43 4B 20 4F 4E 20 55 4E 49 54 20 00 3D 53    LOCK ON UNIT .=S

00D58410  45 4E 54 2C 52 45 43 56 44 2E 20 41 44 44 52 45    ENT,RECVD. ADDRE

00D58420  53 53 20 52 45 47 20 46 41 49 4C 20 54 4F 20 4C    SS REG FAIL TO L

00D58430  4F 41 44 2E 20 55 4E 49 54 20 00 00 3D 45 52 52    OAD. UNIT ..=ERR

00D58440  4F 52 20 53 54 41 54 55 53 20 41 46 54 45 52 20    OR STATUS AFTER

00D58450  41 20 22 43 22 20 50 55 4C 53 45 2E 20 55 4E 49    A "C" PULSE. UNI

00D58460  54 20 00 00 20 41 44 44 52 45 53 53 2B 57 43 2C    T .. ADDRESS+WC,

00D58470  44 49 42 20 52 45 41 44 49 4E 47 2E 20 57 52 49    DIB READING. WRI

00D58480  54 45 20 55 4E 49 54 20 00 00 3D 53 54 41 54 55    TE UNIT ..=STATU

00D58490  53 20 41 46 54 45 52 20 41 20 57 52 49 54 45 2E    S AFTER A WRITE.

00D584A0  20 55 4E 49 54 20 00 00 3D 53 54 41 54 55 53 20     UNIT ..=STATUS

00D584B0  41 46 54 45 52 20 41 20 53 50 41 43 45 2E 20 55    AFTER A SPACE. U

00D584C0  4E 49 54 20 00 00 3D 53 54 41 54 55 53 20 41 46    NIT ..=STATUS AF

00D584D0  54 45 52 20 41 20 45 52 41 53 45 2E 20 55 4E 49    TER A ERASE. UNI

00D584E0  54 20 00 00 3D 53 54 41 54 55 53 20 41 46 54 45    T ..=STATUS AFTE

00D584F0  52 20 57 52 49 54 45 20 45 4F 46 2E 20 55 4E 49    R WRITE EOF. UNI

00D58500  54 20 00 00 4E 4F 20 45 4F 46 20 53 54 41 54 55    T ..NO EOF STATU

And still other needs both bit drop and byte swap.

00D518B0  20 56 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 00 00     V============..

00D518C0  69 5C 0D 0D 48 54 53 49 50 20 4F 52 52 47 4D 41    i\..HTSIP ORRGMA  = [THIS PROGRAM]

00D518D0  47 20 56 49 53 45 47 20 49 55 41 44 43 4E 20 45    G VISEG IUADCN E

00D518E0  4E 49 53 20 54 45 49 54 47 4E 55 20 20 50 44 24    NIS TEITGNU  PD$

00D518F0  4D 47 2E 58 62 00 69 5C 4F 59 20 55 49 57 4C 4C    MG.Xb.i\OY UIWLL

00D51900  42 20 20 45 53 41 45 4B 20 44 4F 54 45 20 54 4E    B  ESAEK DOTE TN

00D51910  52 45 50 20 52 4F 20 54 59 54 45 50 41 20 44 4E    REP RO TYTEPA DN

00D51920  42 20 55 41 20 44 41 52 45 54 46 20 52 4F 45 20    B UA DARETF ROE

00D51930  43 41 00 48 00 62 69 5C 50 20 52 4F 20 54 4F 54    CA.H.bi\P RO TOT

00D51940  42 20 20 45 45 44 49 46 45 4E 2E 44 20 20 4F 46    B  EEDIFEN.D  OF

00D51950  20 52 41 45 48 43 50 20 49 52 54 4E 52 45 50 20     RAEHCP IRTNREP

00D51960  52 4F 20 54 4F 59 20 55 49 57 4C 4C 41 20 53 4C    RO TOY UIWLLA SL

00D51970  00 4F 00 62 69 5C 45 20 54 4E 52 45 41 20 55 20    .O.bi\E TNREA U

00D51980  49 4E 55 51 20 45 52 50 4E 49 45 54 20 52 41 4E    INUQ ERPNIET RAN

00D51990  45 4D 28 20 2E 49 2E 45 24 20 50 4C 2C 54 24 20    EM( .I.E$ PL,T$

00D519A0  50 4C 31 54 20 2C 4C 24 54 50 2E 32 2E 2E 20 29    PL1T ,L$TP.2.. )

00D519B0  00 2E 00 62 69 5C 49 20 20 46 4F 59 20 55 52 50    ...bi\I  FOY URP

00D519C0  53 45 20 53 45 52 55 54 4E 52 46 20 52 4F 54 20    SE SERUTNRF ROT

00D519D0  50 59 20 45 52 4F 42 20 55 41 20 44 48 54 4E 45    PY EROB UA DHTNE

00D519E0  54 20 45 48 50 20 45 52 49 56 55 4F 00 53 00 62    T EHP ERIVUO.S.b

00D519F0  69 5C 56 20 4C 41 45 55 57 20 4C 49 20 4C 45 42    i\V LAEUW LI LEB

00D51A00  52 20 50 45 41 45 45 54 2E 44 20 20 48 54 20 45    R PEAEET.D  HT E

00D51A10  45 44 41 46 4C 55 20 54 4E 45 52 54 20 59 4F 46    EDAFLU TNERT YOF

00D51A20  20 52 48 54 20 45 49 46 53 52 00 54 00 62 69 5C     RHT EIFSR.T.bi\

00D51A30  45 20 54 4E 59 52 49 20 20 53 45 54 4D 52 4E 49    E TNYRI  SETMRNI

00D51A40  4C 41 39 20 30 36 20 30 41 42 44 55 00 2E 00 62    LA9 06 0ABDU...b

Not a full solution, but thought it may help.

...

Thanks,

John Svee

303-877-8979

==============================================================

I have implemented the bit drop using this code (the language is LotusScript, similar to Visual Basic):

Sub HighestOrderBitDropper

    ' This is the actual data block.  Here is where we drop the highest order bit.  And this is how:

                    ' For the FIRST nybble in the two-nybble pair that make up the byte, swap these for these: 

    '8 -> 0

    '9 -> 1

    'A -> 2

    'B -> 3

    'C -> 4

    'D -> 5

    'E -> 6

    'F -> 7

 For n = 1 To Len(BlockHolder)

  If n/2 <> Int(n/2) Then ' this should only find the odd numbered nybbles

   fromlist = Chr(34) + "8" + Chr(34) + ":" + Chr(34) + "9" + Chr(34) + ":" + Chr(34) + "A" + Chr(34) + ":" + Chr(34) + "B" + Chr(34) + ":" + Chr(34) + "C" + Chr(34) + ":" + Chr(34) + "D" + Chr(34) + ":" + Chr(34) + "E" + Chr(34) + ":" + Chr(34) + "F" + Chr(34)

   tolist = Chr(34) + "0" + Chr(34) + ":" + Chr(34) + "1" + Chr(34) + ":" + Chr(34) + "2" + Chr(34) + ":" + Chr(34) + "3" + Chr(34) + ":" + Chr(34) + "4" + Chr(34) + ":" + Chr(34) + "5" + Chr(34) + ":" + Chr(34) + "6" + Chr(34) + ":" + Chr(34) + "7" + Chr(34)

   TheMacro = "@Replace(" + Chr(34) + Mid(BlockHolder,n,1) + Chr(34) + ";" + fromlist + ";" + tolist + ")"   

   TheResult = Evaluate(TheMacro)

   NewBlockHolder = NewBlockHolder + TheResult(0) 

'==============================================================

'And I have implemented the byte swap using this part of the code 

  Else

   NewBlockHolder = NewBlockHolder + Mid(BlockHolder,n,1)

   

''''''start Reverse Endian Operation   

   ThisByte = Mid(NewBlockHolder,Len(NewBlockHolder)-1,2) 

   'I want to capture full bytes, then step through and count those for even/odd.  The byte is only captured on every 

   'even nybble so far, or so it should be.

   ByteOneTwo = ByteOneTwo + 1

   If ByteOneTwo = 2 Then

    SecondByte = ThisByte

    ReverseEndianBlock = ReverseEndianBlock + SecondByte + FirstByte    

    ByteOneTwo = 0

   Else

    FirstByte = ThisByte 

   End If

  End If

  

 Next n

 Print #7, ReverseEndianBlock  'Bit Drop and Byte Swap Block

 Print #8, NewBlockHolder ' Bit Drop Only Block

End Sub

------------------------------------------------------

And, I just found something in the IRIS Migration Toolkit manual on the bottom pf PDF Page 10, that I never would have understood before I went through this "dropping highest order bit" process:

"The data is converted to 7-bit ASCII with the high-order bit set to 0. (This is the opposite of IRIS which sets the high-order bit to 1.)"  

In retrospect, I can see that this paragraph clearly speaks to this topic.