How to patch 11 different firmware and do not go crazy with diversity

If any operation turns into a routine - automate it. Even if you spend more time - but you did not do your routine, but an interesting thing. It was under this sign that instead of simply patching the new 11 versions of rtsp_streamer for cameras from TopSee , I decided to draw an auto-patch. I consider the python to be the ideal language for any knee-pieces - quite concisely, tough enough for readability (although I still manage to make it unreadable). In general, now I will tell you how to learn how to draw auto-patches with the help of a stick and a rope in one evening.

So, the main requirements for scripts on the knee - the maximum match expectations. He must either work or report that something is not right. The main mistake of such scripts is any actions without checking for compliance with expectations. Since otherwise you can not notice that something has changed and requires human intervention.

So, remember what we did last time , and go through the whole path again manually:
1) Ship the file to disassembler
2) Find the function fctnl
3) Pass by calls in search of using fcntl with O_NONBLOCK - we find two functions, makeSocketBlocking and makeSocketNonBlocking

Hidden text

Boolean makeSocketNonBlocking(int sock) { #if defined(__WIN32__) || defined(_WIN32) unsigned long arg = 1; return ioctlsocket(sock, FIONBIO, &arg) == 0; #elif defined(VXWORKS) int arg = 1; return ioctl(sock, FIONBIO, (int)&arg) == 0; #else int curFlags = fcntl(sock, F_GETFL, 0); return fcntl(sock, F_SETFL, curFlags|O_NONBLOCK) >= 0; #endif } Boolean makeSocketBlocking(int sock) { #if defined(__WIN32__) || defined(_WIN32) unsigned long arg = 0; return ioctlsocket(sock, FIONBIO, &arg) == 0; #elif defined(VXWORKS) int arg = 0; return ioctl(sock, FIONBIO, (int)&arg) == 0; #else int curFlags = fcntl(sock, F_GETFL, 0); return fcntl(sock, F_SETFL, curFlags&(~O_NONBLOCK)) >= 0; #endif }

4) We are looking for the sendPacket () function in the code (we know from the past that debug printfs are entered into it, for which it is easy to find)

Hidden text

 Boolean RTPInterface::sendPacket(unsigned char* packet, unsigned packetSize) { Boolean success = True; // we'll return False instead if any of the sends fail for (tcpStreamRecord* streams = fTCPStreams; streams != NULL; streams = streams->fNext) { if (!sendRTPOverTCP(packet, packetSize, streams->fStreamSocketNum, streams->fStreamChannelId)) { printf("%s(): ", "sendPacket"); printf("sendRTPOverTCP failed, sock: %d, chn: %d\r\n", streams->socket, streams->fStreamChannelId); success = False; } } return success; }

5) Patch function

Hidden text

 Boolean RTPInterface::sendPacket(unsigned char* packet, unsigned packetSize) { Boolean success = True; // we'll return False instead if any of the sends fail for (tcpStreamRecord* streams = fTCPStreams; streams != NULL; streams = streams->fNext) { makeSocketBlocking(streams->socket); Boolean res = sendRTPOverTCP(packet, packetSize, streams->fStreamSocketNum, streams->fStreamChannelId); makeSocketNonBlocking(streams->socket); if (!res) { success = False; } } return success; }

So, that's all we need to automate so that launched and received. Moreover, in different versions of the firmware used somewhere fcntl, somewhere fcntl64; Depending on the compiler options, different registers are used, small other differences in the translated code are observed.
')
So, let's start writing our script. Understandably, since the patch will be different versions, the name of the file being patched must be passed as an argument. So the script started:

 import sys fname = sys.argv[1]

Since we are making a script for ourselves, the files are small (~ meter weighing), there is a lot of memory, so we will not bother with running through the file - we will load everything into memory at once:

 f = open(fname, "r+b") f.seek(0, 2) size = f.tell() f.seek(0, 0) fw = f.read(size) f.close()

Let's start looking for the makeSocketBlocking / makeSocketNonBlocking functions. The fcntl functions are used a lot where, so the real hook is O_NONBLOCK (= 0x800). On the other hand, these are library functions that nobody touches, they go in a row, you can simply find the functions "as is":

 .text:0003C554 makeSocketBlocking .text:0003C554 10 40 2D E9 STMFD SP!, {R4,LR} .text:0003C558 03 10 A0 E3 MOV R1, #F_GETFL ; cmd .text:0003C55C 00 20 A0 E3 MOV R2, #0 .text:0003C560 00 40 A0 E1 MOV R4, R0 .text:0003C564 32 39 FF EB BL fcntl .text:0003C568 04 10 A0 E3 MOV R1, #F_SETFL ; cmd .text:0003C56C 02 2B C0 E3 BIC R2, R0, #O_NONBLOCK .text:0003C570 04 00 A0 E1 MOV R0, R4 ; fd .text:0003C574 2E 39 FF EB BL fcntl .text:0003C578 00 00 E0 E1 MVN R0, R0 .text:0003C57C A0 0F A0 E1 MOV R0, R0,LSR#31 .text:0003C580 10 80 BD E8 LDMFD SP!, {R4,PC} .text:0003C580 ; End of function makeSocketBlocking .text:0003C584 makeSocketNonblocking ; CODE XREF: sub_43524+40 p .text:0003C584 ; .text:00043608 p ... .text:0003C584 10 40 2D E9 STMFD SP!, {R4,LR} .text:0003C588 03 10 A0 E3 MOV R1, #3 ; cmd .text:0003C58C 00 20 A0 E3 MOV R2, #0 .text:0003C590 00 40 A0 E1 MOV R4, R0 .text:0003C594 26 39 FF EB BL fcntl .text:0003C598 04 10 A0 E3 MOV R1, #4 ; cmd .text:0003C59C 02 2B 80 E3 ORR R2, R0, #0x800 .text:0003C5A0 04 00 A0 E1 MOV R0, R4 ; fd .text:0003C5A4 22 39 FF EB BL fcntl .text:0003C5A8 00 00 E0 E1 MVN R0, R0 .text:0003C5AC A0 0F A0 E1 MOV R0, R0,LSR#31 .text:0003C5B0 10 80 BD E8 LDMFD SP!, {R4,PC} .text:0003C5B0 ; End of function makeSocketNonblocking

We copy this opcodes in the forehead, and replace all the parameters that differ from the version, address, compiler settings and so on.

 blockMask = """ ; makeSocketBlocking mm ?? ?? 2D E9 ; STMFD SP!, .... 03 10 A0 E3 ; MOV R1, #3 ; cmd 00 20 A0 E3 ; MOV R2, #0 00 ?? A0 E1 ; MOV Rx, R0 ; save fd ?? ?? FF EB ; BL fcntl 04 10 A0 E3 ; MOV R1, #4 ; cmd 02 2B C0 E3 ; clear O_NONBLOCK ?? 00 A0 E1 ; restore fd ?? ?? FF EB ; BL fcntl 00 00 E0 E1 ; MVN R0, R0 A0 0F A0 E1 ; MOV R0, R0,LSR#31 ?? ?? BD E8 ; LDMFD SP!, .... mm ; makeSocketNonblocking ?? ?? 2D E9 03 10 A0 E3 00 20 A0 E3 00 ?? A0 E1 ?? ?? FF EB 04 10 A0 E3 02 2B 80 E3 ?? 00 A0 E1 ?? ?? FF EB 00 00 E0 E1 A0 0F A0 E1 ?? ?? BD E8 """

I additionally put labels ("mm") to the beginning of the functions; all non-opcodes commented out ";" and left it in the code as it is (for the future, to remember what is what).
Now we need to turn this line into a search mask. Of course, I’m not ready to search manually at all, so we’ll search through regexps, since they are debugged and optimized, don’t indulge in the most. And we read the entire file in memory as one big line - which is also convenient. Therefore, we write a function that converts this mask to regexp:

 import re def maskToRegex(mask): mask = re.sub( ";.*$", "", mask, flags=re.MULTILINE) mask = re.sub( "\s+", "", mask, flags=re.MULTILINE) masks = re.findall( "..", mask) rgx = "" for m in masks: if m == "??": rgx += "." elif m == "mm": rgx += "()" else: rgx += "\\x"+m return rgx

The behavior is simple - we delete all comments (everything from; to the end of the line), remove all spaces, cut the remaining characters into pairs, and see - if it is ?? - then we replace it with a point (any character), if “mm” we insert a label for which we will memorize the position, otherwise we generate a character code (by assigning “\ x” to this pair).

So, we have a mask and we can get a retex from it, let's finally find these functions:

 ### 1. Find offset of makeSocketBlocking and makeSocketNonblocking makeBlock = None makeNonBlock = None for find in re.finditer(maskToRegex(blockMask), fw, re.DOTALL): if makeBlock is None and makeNonBlock is None: makeBlock = find.start(1) makeNonBlock = find.start(2) print "Found makeNonBlock at ", hex(makeNonBlock) print "Found makeBlock at ", hex(makeBlock) else: print "Non-unqiue makeNonBlock/makeBlocking functions found" break if makeBlock is None or makeNonBlock is None: print "makeNonBlock/makeBlocking functions not found"

In this code there are two important points. First, global, meeting expectations. We check that only one block of code has fallen under this mask, and that it has really fallen. At the time of debugging, we add a printout of the offset of the places found. Secondly, it is important not to forget about re.DOTALL, so that any byte should fall under the dot, we work with a binary string.

So, now we need to find the sendPacket function. Let's look at the disassembly:

 .text:0006C9A0 SendPacket ; CODE XREF: sub_69FB8+144 p .text:0006C9A0 ; .text:0006D144 p ... .text:0006C9A0 F0 4F 2D E9 STMFD SP!, {R4-R11,LR} .text:0006C9A4 00 60 A0 E1 MOV R6, R0 ..... .text:0006CB78 C3 FF FF 1A BNE loc_6CA8C .text:0006CB7C E2 FF FF EA B loc_6CB0C .text:0006CB7C ; End of function SendPacket .text:0006CB80 BC 92 0A 00 off_6CB80 DCD aS_10 ; DATA XREF: SendPacket+7C r, SendPacket+F0 r .text:0006CB80 ; "%s(): " .text:0006CB84 E4 6B 0A 00 off_6CB84 DCD aSendpacket ; DATA XREF: SendPacket+80 r, SendPacket+F4 r .text:0006CB84 ; "sendPacket" .text:0006CB88 00 9B 0A 00 off_6CB88 DCD aSendrtpovert_0 ; DATA XREF: SendPacket+98 r .text:0006CB88 ; "sendRTPOverTCP failed, sock: %d, chn: %"... .text:0006CB8C 2C 9B 0A 00 off_6CB8C DCD aRemovestreamso ; DATA XREF: SendPacket+110 r .text:0006CB8C ; "removeStreamSocket, sock: %d, chnid: %d"...

Aha, the link to this line lies right after the function code. So, to find a function, we need to: find the line with the name of the function (thanks to the debugging macros, it is a separate independent line), convert the offset to the address, find this address, in the code, from this address unwind up to the STMFD SP! ..., LR}.
Immediately the question arises - to find the string is not a problem, but how to convert the offset in the file into a virtual address? Do not parse the file manually. This is where all-powerful Google comes to the rescue: there is a pyelftools package. So "pip install pyelftools", and smoke the attached documentation. There is nothing useful there. OK, we stupidly climb into the elffily.py file and see what is tasty there. We find there a function that does the reverse task - it finds an offset in the virtual address:

  def address_offsets(self, start, size=1): """ Yield a file offset for each ELF segment containing a memory region. A memory region is defined by the range [start...start+size). The offset of the region is yielded. """ end = start + size for seg in self.iter_segments(): if (start >= seg['p_vaddr'] and end <= seg['p_vaddr'] + seg['p_filesz']): yield start - seg['p_vaddr'] + seg['p_offset']

Everything is clear, with a flick of the wrist these pants turn into elegant shorts:

 #  f.close(),      from elftools.elf.elffile import ELFFile f.seek(0, 0) Elf = ELFFile(f) def offToVA(offset): for k in Elf.iter_segments(): if offset >= k['p_offset'] and offset <= k['p_offset']+k['p_filesz']: return k['p_vaddr']+(offset-k['p_offset'])

Now we can find the string and its place of use:

  s = "sendPacket" ## Find string itself offStr = re.findall(s+"\x00", fw) if len(offStr)==1: offStr = re.search(s+"\x00", fw) offStrVA = offToVA(offStr.start(0)) print "offStr["+s+"] =", hex(offStrVA) elif len(offStr)==0: print s, "string marker not found" else: print "Too many", s, "string markers found"

Then, due to laziness, I used another method of checking expectations - I just try to find everything first, and I think that everything is ok if she is alone. Otherwise, I report an error. The same method is not convenient, + unnecessary searches, but I personally understand, so I will not repeat the experiments with finditer.
Well, now we find the offset where the string is used:

  ## Find offset to string reStrLink = "\\x%02X\\x%02X\\x%02X\\x%02X" % ( (offStrVA)%256, (offStrVA/256)%256, (offStrVA/256/256)%256, (offStrVA/256/256/256)%256 ) offLink = re.findall(reStrLink, fw) if len(offLink)==1: offLink = re.search(reStrLink, fw) offLink = offLink.start(0) if DEBUG: print "offLink["+s+"] = ", hex(offToVA(offLink)) return offLink else: print "Can't find usage of", s

As you already guessed, instead of s = "sendPacket" , def findStringLink(s): is actually written def findStringLink(s): since we will need to find the function more than once.
Well, now we need to find another beginning of the function - let's walk back 4 bytes in search of STMFD SP !, {..., LR}. Because of laziness, I limited myself to searching for STMFD SP !, {...} (so as not to analyze the bits). The reason is that if you don’t find the beginning of the function, the search will still break further, which I’ll find out, and then I can decide how to fix it better.

 # Find previous function begin offset (nearest STMFD SP!, {...} instruction) def findFuncBegin(offset, maxLen = 0x1000): maxStart = max(0, offset-maxLen) offset -= 4 while offset > maxStart: if fw[offset+2:offset+4]=="\x2D\xE9": return offset offset -= 4 return None

So, we have everything we need, finally we will find the sendPacket function:

 ### 2. Find sendPacket function sendPacketEnd = findStringLink("sendPacket") sendPacketStart = findFuncBegin(sendPacketEnd) if sendPacketStart is not None: print "sendPacketStart = ", hex(offToVA(sendPacketStart)) else: print "Can't find start of sendPacket"

Now we are closer to the most interesting part - we need to find a loop inside sendPacket, and make sure that it is it. We make up the mask as we learned earlier:

 sendLoopMask = """ ?? 00 00 EA ; B loopBody ; --------------------------------------------------------------------------- mm ;loopNext ; CODE XREF: SendPacket+74j ?? ?? ?? E5 ; LDR R4, [R4,#4] (or R5) 00 00 ?? E3 ; CMP R4, #0 (or R5) ?? 00 00 0A ; BEQ loc_6CA74 ;loopBody ; CODE XREF: SendPacket+4Cj mm ; ; SendPacket+D0nj ?? ?? A0 E1 ; MOV R3, R4 (or R5) ?? ?? A0 E1 ; MOV R1, R5 ?? ?? A0 E1 ; MOV R2, R7 ?? ?? A0 E1 ; MOV R0, R6 mm ?? ?? ?? EB ; BL SendRTPOverTCP 00 00 50 E3 ; CMP R0, #0 F5 FF FF AA ; BGE loopNext """

But we need to check that the BL inside the loop is exactly to SendRTPOverTCP, otherwise it either changed the function strongly, or the loop has already been patched by us, therefore we will also find SendRTPOverTCP:

 ### 3. Find sendRTPOverTCP function sendRTPOverTCPStart = findFuncBegin(findStringLink("sendRTPOverTCP")) if sendRTPOverTCPStart is not None: print "sendRTPOverTCPStart = ", hex(offToVA(sendRTPOverTCPStart)) else: print "Can't find start of sendPacket"

So, but we should be able to check whether the link is there. All transitions in ARM are not absolute, but relative, plus the addresses are given by calculating +2 instructions from the current one, and even specified in the quanta of instructions (that is, divided by 4). In general, the opcode is drawn something like this :
TargetAddr = Opcode*(4 bytes/word) + CurAddr + 8

As a result, we obtain such functions, for calculating the address where some instruction refers, and for calculating which operand will be instructions for a certain address to go to where it should be:

 def BinArg(off): return ord(fw[off])+ord(fw[off+1])*256+ord(fw[off+2])*256*256 def ArgToBin(arg): return chr(arg%256)+chr(arg/256%256)+chr(arg/256/256%256) def cmdTargetOffset(cmdoff): d1 = BinArg(cmdoff) if d1 >= 0x800000: d1 -= 0x1000000 return (cmdoff+(d1+1)*4+4) def cmdTargetArg(cmdoff, target): d1 = (target - (cmdoff+4))/4 - 1 if d1 < 0: d1 += 0x1000000 return d1

Now that the bricks are set aside, let's build another bulkhead:

 ### 4. find loop in sendPacket sendPacketLoopRx = maskToRegex(sendLoopMask) sendPacketLoop = re.findall(sendPacketLoopRx, fw, re.DOTALL) if len(sendPacketLoop)==1: sendPacketLoop = re.search(sendPacketLoopRx, fw, re.DOTALL) sendPacketLoopNext = sendPacketLoop.start(1) sendPacketLoopBL = sendPacketLoop.start(3) sendPacketLoop = sendPacketLoop.start(2) if DEBUG: print "sendPacket loop at ", hex(offToVA(sendPacketLoop)) elif len(sendPacketLoop)==0: print "Loop inside sendPacket not found" else: print "Non-unqiue loops masks for sendPacket found" ## 4.1. check that loop link is really to sendRTPOverTCP if cmdTargetOffset(sendPacketLoopBL) != sendRTPOverTCPStart: print "Loop's first call is not sendRTPOverTCP" if cmdTargetArg(sendPacketLoopBL, sendRTPOverTCPStart)!=BinArg(sendPacketLoopBL): print "BUG! cmdTargetArg inconsistent with cmdTargetOffset!" if ArgToBin(cmdTargetArg(sendPacketLoopBL, sendRTPOverTCPStart)) != fw[sendPacketLoopBL:sendPacketLoopBL+3]: print "BUG! ArgToBin inconsistent with cmdTargetOffset!"

So, we found a loop, checked that the BL in it refers specifically to sendRTPOverTCP, and at the same time checked the address functions that they are consistent. All this allows you to protect yourself from typos and get rid of unnecessary coverages with function tests — we are writing a script, not a software product, therefore only the necessary maximum of gestures.

But in order to patch, we need to add 6 new instructions, and for this we need a place. Since we have two extra prints in the cycle, as we already know, we use them. The first print 5 instructions, it means you will have to wipe both. To do this, they will have to find and make sure that they:

 ### 5. Find next two printfs printf1 = sendPacketLoopBL+4 while printf1 < sendPacketEnd: if fw[printf1+3]=="\xEB": printfStart = cmdTargetOffset(printf1) break printf1 += 4 printf1 += 4 printf2 = printf1 + 4 while printf2 < sendPacketEnd: if fw[printf2+3]=="\xEB": if cmdTargetOffset(printf2) != printfStart: print "ERROR! After loop not two printfs!" break printf2 += 4 printf2 += 4 if (printf1-sendPacketLoop)/4-7 != 5: print "WARN! First printf not 5 instructions" if (printf1-sendPacketLoop)/4-7 > 5: printf2 = printf1 # no need to cleanup 2nd printf

Again, we go the simplest way - we take the two closest BL after BL sendRTPOverTCP, check that they both refer to the same function, and check their sizes. If anything goes wrong, we will immediately curse. If everything is in line with expectations, then everything is fine.

Well, now put all this together, add three buckets of foolishness, and generate a patch:

 ### 6. Generate new loop body PatchSendPacket = "" ## 6.1. LDR R0, Socket(Rx#8) ldrSock = "\x08\x00"+fw[sendPacketLoopNext+2]+"\xE5" PatchSendPacket += ldrSock ## 6.2. BL makeSocketBlocking tgtSocketBlock = cmdTargetArg(sendPacketLoop+len(PatchSendPacket), makeBlock) PatchSendPacket += ArgToBin(tgtSocketBlock)+"\xEB" ## 6.3. Copy 4 MOVs PatchSendPacket += fw[sendPacketLoop:sendPacketLoopBL] ## 6.4. BL sendRTPOverTCP tgtSendRTPOverTCP = cmdTargetArg(sendPacketLoop+len(PatchSendPacket), sendRTPOverTCPStart) PatchSendPacket += ArgToBin(tgtSendRTPOverTCP)+"\xEB" ## 6.5. STMFD SP!, {R0} PatchSendPacket += "\x01\x00\x2D\xE9" ## 6.6. LDR R0, Socket PatchSendPacket += ldrSock ## 6.7. BL makeSocketNonBlocking tgtSocketNonBlock = cmdTargetArg(sendPacketLoop+len(PatchSendPacket), makeNonBlock) PatchSendPacket += ArgToBin(tgtSocketNonBlock)+"\xEB" ## 6.8. LDMFD SP!, {R0} PatchSendPacket += "\x01\x00\xBD\xE8" ## 6.9. CMP R0, #0 PatchSendPacket += "\x00\x00\x50\xE3" ## 6.A. BGE loopNext tgtLoopNext = cmdTargetArg(sendPacketLoop+len(PatchSendPacket), sendPacketLoopNext) PatchSendPacket += ArgToBin(tgtLoopNext)+"\xAA" ## 6.B. Fill up to printf2 with NOPs Nops = (printf2 - (sendPacketLoop + len(PatchSendPacket))) / 4 PatchSendPacket += "\x00\x00\xA0\xE1" * Nops ## 6.C. Save generated patch patches = [] patches.append( (sendPacketLoop, PatchSendPacket) ) print "Successfully patched"

Now that we have all the patches (as long as there is a whole one, but first and foremost), we will generate a new file:

 ### FIN: save patched file if True: f = open(fname+".fixed", "w+b") patches.sort() last = 0 for p in patches: f.write( fw[last:p[0]] ) f.write( p[1] ) last = p[0]+len(p[1]) f.write(fw[last:]) f.close()

So, using a stick, a rope, a python and a gram of the brain, we easily and easily patched all 11 streamers I needed at once, and the time was spent one evening, which is approximately equal to the time it would take to manually patch them all. That's just the next update, no longer need to spend time at all!

 for k in `(cd todo; ls -1)`; do g=$(echo $k | perl -pe 's/.*?([TV][^-]+).*?-V(2[^_]+).*/$1_$2/'); mv todo/$k .; ../repack/unpack.sh $k; cp $k.unpack/root/opt/topsee/rtsp_streamer rtsp_streamer_$g; done for k in rtsp_streamer_*.[0-9]; do python tcpfix.py $k; done ...

Ps: to everyone who needs something else from these firmware, put the scripts in a more convenient place - on the githab .

Source: https://habr.com/ru/post/228317/

All Articles

How to patch 11 different firmware and do not go crazy with diversity

More articles: