1 (edited by SheldonCooper 2011-05-31 04:44:10)

Topic: Potential bug + fix in aes.c AesDecrypt function

I have been using the sniffer program for some personal learning, I have encountered the following scenario:
the function AesDecrypt from ctaocrypt/aes.c was called with an aes (type AES*) param whose "rounds" member was zero,
aes->rounds == 0
later in the code, it states:
if(r > 7)

afterwards we go on a for loop which will take place r times (breaking from the loop in the middle of the r-th time). in the loop, we raise rk (which is initialized to point to an array of size 60) by 8 and lower r by 1, per loop. if r was zero before entering the loop, we will effectively have r=0 for the first half of the first time in the loop, then, since r is of type word32 (=unsigned int), we will make r be (2^32)-1 after the r-- operation, thus resulting in a an almost infinite loop, during which rk will definitely exceed the bounds of the array it points to, which will result in a segmentation fault.

I am not sure whether the traffic i was testing is considered "correct", maybe I accidentally  stumbled upon some corrupted packets, but anyhow,
i think it could be wise to change the return condition before entering the loop to:
if(r > 7 || r == 0)

hope this helps someone,



Re: Potential bug + fix in aes.c AesDecrypt function

Hi Sheldon,

Thanks for the bug fix.  We've made your suggested change and committed it to our GitHub repository.