Dealing with partial code tampering on external memory

[ajie_dirgantara]

Well, not really as if the customer itself wanted their product to be break if it is tampered, instead of working but not as designed. Take a sample of rented product. Our customers (A) is a service company, want their product to be working as designed as it is, when they rent it to their customers (B). But then (B) could alter something which harmful to (A) until the rent is due and (B) restore the original codes, hence leaves (A) unnoticed. If only we could have a way to make (A) knew that (B) is alter some code when this product is being rented. Broken product is considered good in this case, because (B) must go back to (A) to return the product.

In that case I would recommend re-flashing the original firmware every time the item is returned before renting it again.

How can re-flashing the original firmware solve the problem? in above example even (B) restore original firmware so they won't get caught by (A).

[ajie_dirgantara]

A forum search will show that this topic has been extensively discussed. IIRC the conclusion was that the best that can be done is to implement the application as frozen bytecode stored in the device's Flash memory. If an attacker has physical access to the device there is little to stop them from interrogating the stored bytecode or installing a firmware build of their own.

I still put hope on the "little to stop" words, as this would probably open some methods to make it non-trivial for people, hence not worth the effort to do so.

[deshipu]

But then you are not selling water meters. I can go to a shop and buy a water meter, open it, look inside, disassemble it, reassemble again, etc., and there is nothing that prevents me from doing that with the property I have bought. However, I can't use that to bill my water consumption, because the water company would not trust it — they only want water meters that they have installed and verified correct, and they will use physical seals to "prove" that they haven't been tampered with. Of course those seals can be (and routinely are) easily worked around, as you can hear from anybody who has ever worked as a plumber, but that's beside the point.

If you are using your pyboard like that, you can do exactly the same thing: seal it physically without access to the USB port or debugging pins, and require an authorized employee to perform any servicing or upgrades, after which they have to re-seal the box. Which actually makes sense, because if you are using some sensors there, they could be tampered with without touching the software too, so you need a way to protect them anyways.

[deshipu]

Well, not really as if the customer itself wanted their product to be break if it is tampered, instead of working but not as designed. Take a sample of rented product. Our customers (A) is a service company, want their product to be working as designed as it is, when they rent it to their customers (B). But then (B) could alter something which harmful to (A) until the rent is due and (B) restore the original codes, hence leaves (A) unnoticed. If only we could have a way to make (A) knew that (B) is alter some code when this product is being rented. Broken product is considered good in this case, because (B) must go back to (A) to return the product.

In that case I would recommend re-flashing the original firmware every time the item is returned before renting it again.

How can re-flashing the original firmware solve the problem? in above example even (B) restore original firmware so they won't get caught by (A).

I'm sorry, I didn't read your description carefully, I assumed the company is renting it and doesn't want the next customer to receive a faulty device.

In the case you are describing, one can't help but wonder what would the "something harmful to the company" be, that could otherwise not be detected. Do you mean something like a voodoo curse, or thinking sinful thoughts? Can you provide an example? The example with water meters that you gave doesn't apply — the company has their own water meters at the pump station, and they can easily see when their meters don't match with yours, and investigate. Same applies to pretty much any physical goods you could sell.

[ajie_dirgantara]

In that case I would recommend re-flashing the original firmware every time the item is returned before renting it again.

How can re-flashing the original firmware solve the problem? in above example even (B) restore original firmware so they won't get caught by (A).

I'm sorry, I didn't read your description carefully, I assumed the company is renting it and doesn't want the next customer to receive a faulty device.

In the case you are describing, one can't help but wonder what would the "something harmful to the company" be, that could otherwise not be detected. Do you mean something like a voodoo curse, or thinking sinful thoughts? Can you provide an example? The example with water meters that you gave doesn't apply — the company has their own water meters at the pump station, and they can easily see when their meters don't match with yours, and investigate. Same applies to pretty much any physical goods you could sell.

I just being thoughtful about the product and yet to decide what to develop.

I think the water company is one of good samples. And no, believe me for this, they didn't have their own water meters at the pump station which directly reflect single meters on endpoint, The meter they had in their pump station is large meter which then distributed to multiple endpoint, and they didn't have the time to check one by one for any tamper on thousand connections. The loss is almost there every time (even using mechanical water meter) and the most feasible thing to do is reduce the loss by make their water meter as accurate as possible.
To investigate how people consume water... well, how do we investigate it if we can't trust the meter? That is why the meter must be tamper-proof, or at least detect tamper and report it upstream (if connected as IoT devices).

I believe this kind of thing could be mitigate using what is available in micropython. It may not perfect, but at least we could shoo away 50% harmful intentions, and let other 50% done by any other means of protections (mechanically maybe?).

[ajie_dirgantara]

But then you are not selling water meters. I can go to a shop and buy a water meter, open it, look inside, disassemble it, reassemble again, etc., and there is nothing that prevents me from doing that with the property I have bought. However, I can't use that to bill my water consumption, because the water company would not trust it — they only want water meters that they have installed and verified correct, and they will use physical seals to "prove" that they haven't been tampered with. Of course those seals can be (and routinely are) easily worked around, as you can hear from anybody who has ever worked as a plumber, but that's beside the point.

If you are using your pyboard like that, you can do exactly the same thing: seal it physically without access to the USB port or debugging pins, and require an authorized employee to perform any servicing or upgrades, after which they have to re-seal the box. Which actually makes sense, because if you are using some sensors there, they could be tampered with without touching the software too, so you need a way to protect them anyways.

.

Another problem with hardware protection is.. cost. E.g for water meter (produced in volumes) this would be hurt a lot, while provide firmware protection is none to zero cost (beside developer cost ofc). Again, this kind of matter is a trade off, but why invest in hardware-high-security if we could provide 50-70% security using firmware-based protection? As long as it is hard to do, easy to detect, high risk to break the device when tampering, it is good enough.

[deshipu]

Yes, the meters at the pump station measure consumption of many customers. And yes, there are always discrepancies — those are included in the budget, and ignored as long as they are not too big. But as soon as someone starts stealing water (or there is some other problem causing the discrepancy), they can see it, investigate and fix it. This is when they go around checking the seals on the meters.

Note that the seals are added on the meters, and the meters don't have any "anti-tampering" measures built into themselves. In particular, they don't break down when you break the seal, and don't cut the flow of the water, and anything like that. The only result of tampering is a broken seal, which you can in turn verify when you detect discrepancies. This is a great design, because while the seals give them a way to check for tampering, the devices can remain simple, robust, fault-proof and cheap. There is no risk of some unanticipated event triggering the safeguards and leaving the customer without water. In the worst case, they can even break the seals and fix or replace the meter, and request re-sealing later on.

The solutions you are suggesting would compromise the devices, make them more unreliable and prone to failure, and you could actually be sued for that in many countries for intentionally making the product you are selling break (there were cases like that).

I still think that a physical seal is the simplest, cheapest, most robust and most socially accepted approach to this problem.

[ajie_dirgantara]

Based on the discussion above, I am agreed that hardware level protection is a "killer" feature. There won't be any discussion forward if it is implemented.

But, as a *cost effective* and *good enough* solution, provide that the device has :

- MCU
- Battery
- Switch
- Codes on external storage

Then again I'll repeat my first question, I'll implement :

1. Enable JTAG fuse on MCU
2. Disable REPL
3. Compile all scripts into .mpy
4. Enable hash256 checking regularly on the .mpy files at external storage and compare it with hash value pre-installed in internal flash memory, if not equal will erase any .mpy files on external memory, and set tamper flag on internal memory.
5. Detect tamper using switch at device cover connected to I/O, once detected will erase any .mpy files on external memory, and set tamper flag on internal memory.

The questions still remain, is the methods above is *good enough* or there is still any hole that is easy to exploit?

[ajie_dirgantara]

Yes, the meters at the pump station measure consumption of many customers. And yes, there are always discrepancies — those are included in the budget, and ignored as long as they are not too big. But as soon as someone starts stealing water (or there is some other problem causing the discrepancy), they can see it, investigate and fix it. This is when they go around checking the seals on the meters.

Note that the seals are added on the meters, and the meters don't have any "anti-tampering" measures built into themselves. In particular, they don't break down when you break the seal, and don't cut the flow of the water, and anything like that. The only result of tampering is a broken seal, which you can in turn verify when you detect discrepancies. This is a great design, because while the seals give them a way to check for tampering, the devices can remain simple, robust, fault-proof and cheap. There is no risk of some unanticipated event triggering the safeguards and leaving the customer without water. In the worst case, they can even break the seals and fix or replace the meter, and request re-sealing later on.

The solutions you are suggesting would compromise the devices, make them more unreliable and prone to failure, and you could actually be sued for that in many countries for intentionally making the product you are selling break (there were cases like that).

I still think that a physical seal is the simplest, cheapest, most robust and most socially accepted approach to this problem.

There won't be any cut flow of the water as the meter is just measuring device. It just flagged as tamper because it know the code is tampered (see the methods above using hashlib).

For physical seal, yes it is already there. But then it only provide protection for physical enclosure. People now are smart enough to break enclosure without break the seal and leave no trace afterwards (at least not at a glance by repaint, reshape).

I don't agree with it become unreliable and prone to failure just by adding tamper-proof feature. If by design, it will only break if it is tampered, then it will definitely break if tampered and it will work just as fine if it work as it should work.

[deshipu]

The questions still remain, is the methods above is *good enough* or there is still any hole that is easy to exploit?

We don't know, because we don't know your case, the audience, the prices involved, the functionality in question and how tampering would be harmful in the first place. All this information would be necessary to formulate a threat model, and *then* you can *start* to think about the security features that are necessary.

I don't agree with it become unreliable and prone to failure just by adding tamper-proof feature. If by design, it will only break if it is tampered, then it will definitely break if tampered and it will work just as fine if it work as it should work.

Bugs and faults are very rarely inserted into products on purpose, more often they happen as a result of oversight, unanticipated situation, or simply a programmer error. So while "by design" it would only break when tampered with, in reality it will probably break in a lot of more cases. You have to be aware that you are basically booby-trapping your product, and that it's illegal in many countries to do so.