Physical Countermeasures
We've already discussed seals, but one obvious countermeasure
is to encase the entire memory chip in plastic/epoxy. This
would make it dramatically harder to access the memory
chip. One concern I have about this is heat: were these
chips designed to operate without any cooling. That seems
like a question that could be experimentally answered.
I think you'd want to use transparent epoxy here, to prevent
an attacker from drilling in, access the memory chip,
and covering it over, maybe with a small piece of plastic
to permit future access. I also had an anonymous correspondent suggest encasing
the entire unit in epoxy, but at most this would be
the circuit board, since the device has buttons and the
like; this would of course make the heat problem worse.
Cryptographic Countermeasures
Another possibility would be to extend the cryptographic checksum
technique I suggested to deal with the dishonest display. At
the end of the election when the totals are recorded the
CPU writes a MAC into the memory over all the votes
(however recorded) as well as writing a MAC over the totals.
It then erases the per-election key from memory (by overwriting
it with zeros). This makes post-election rewriting
attacks much harder—the attacker would need to also know the
per-election key (which requires either insider information
or access to the machine between setup and the election) and the
per-machine key, which requires extensive physical access.
I think it's plausible to argue that the machine can be secured
at least during the election and potentially before it.
Note that this system could be made much more secure by having
a dedicated memory built into the CPU
for storage of the per-unit key, but that
would involve a lot more reengineering than I'm suggesting
here.

There are materials developed expressly for the purpose of encasing electronics; this is known as "potting". It is my understanding that heat insulation isn't usually as much of a concern as the rate of expansion in response to changes in temperature. (This can be very bad for surface-mount PCBs.)