1

(1 replies, posted in wolfSSL)

Hey All,
I was trying to run the snakeoil2.pcap example from wireshark (decrypt ssl with a known private server key). it appears that the sniffer is having some trouble with this file - intially missing the client hello, and after some modificatinos reachingthe DriveKeys function in keys.c where where the "rounds" parameter was calculated to be 9.
this results in a call to SetPrefix (also found in keys.c) with idx value > 6 - which is not a covered as part of the switch case over there.
1. Is 9 a reasonable value to obtain for "rounds"?
2. should the snakeoil2 example be decrypted at all? or is it unsupported for whichever reasons? (which?)

you can obtain the sample capture file and matching key file by downloading snakeoil2_070531.tgz from
http://wiki.wireshark.org/SampleCaptures
or direct link to the download:
http://wiki.wireshark.org/SampleCapture … 070531.tgz


thanks in advance,

DanC

thanks for your reply. If not flawed, my experiment shows that replacing the define for XMALLOC(s,h,t) with malloc(s) and respectively for XFREE with free(p) resulted in steady memory consumption as opposed to suspected memory leakage. I agree this is strange. I will conduct it again and let you know for sure. in the mean time - what is the meaning of ctx->heap when passed to XMALLOC?


one more thing about my method: I run traffic through the application (a lot of traffic I recorded over months from my website), and watch the TOP on my machine. I define memory leakage when linux's TOP does not go down to "base levle" after i finish transmitting traffic and RemoveStaleSessions is called (I added a way for me to call this method externally). I also exposed counters for the number of sessions and packet buffers allocated and freed - so this is certainly not the issue (all are eventually zeroed).

do you find this method reasonable? can you, from your experience, rely on TOP in such a manner?

thanks in advance,
Dan

Albeit the clear lack of interest  wink , I will update on my current results:
I performed the following:
in ssl.c/wolfSSL_new(WOLFSSL_CTX* ctx) I change:

ssl=(WOLFSSL*)XMALLOC(size(WOLFSSL),ctx->heap,DYNAMIC_TYPE_SSL)

with

ssl=(WOLFSSL*)malloc(sizeof(WOLFSSL))

this - for some reason, appears to have solved the issue.

anyone cares to shed some light on XMALLOC's operation?

i'm buffled...

thanks,
Dan

Hey again,
I have redesigned my experiment and got the following results:
1. I added counters for the number of sessions and packet buffers created - by the end of the run (after enough time without traffic) - all sessions and packets were accounted for being freed
2. it appears that the following is true:
I have skipped the creating of the session->sslServer and session->sslClient, and returned just before CheckPreRecord in sniffer.c/ssl_decodePacket.
3. also, this happens without ProcessMessage - which rules out a pretty large section of code to inspect.

that way, it appears that the overall memory consumption is much lower, but more importantly, returns back to base level after done transmitting traffic (a large recording, so i can see actual memory shifts...).

this turns my suspicion towards SSL_new and SSL_free - is it possible that XMALLOC and XFREE aren't performing "traditional" malloc/free operations, but some sort of pooling, I was trying to understand from the code but failed to reach certainty.

any help will be greatly appreciated.

thanks,
Dan

5

(3 replies, posted in wolfSSL)

Hey all,
Just to be clear - If there is any sort of memory pooling, I would appreciate it if you could let me know...


thanks ,
Dan

6

(3 replies, posted in wolfSSL)

Update:
while I still suspect something, my experiment was flawed. I will update if anything comes up.


thanks and sorry for the false alarm.

Dan

7

(3 replies, posted in wolfSSL)

Hey all,
I noticed my sniffer application demonstrated the following behavior:
I commented out the "ProcessMessage" function, so essentially all i do is add sessions and remove them.
if I comment out (replace by NULL) the calls to SSL_new for session->sslClient/sslServer in the CreateSession method, I see the memory is dead on. however, if I enable the SSL_new in CreateSession, I notice the memory very slowly rising, and does not appear to be going down to the base level when I stop pushing traffic.

1. is it possible that there is a memory leak in SSL_free / SSL_new?
2. if not, are you using memory pooling with the SSL* objects? clearly I can look in the code, but I want to be sure - I ask because that could account for what I am seeing


thanks in advance,
Dan

you are right, this patch should be under the locked region. thanks for patching!

Dan

thanks for your reply - however, I already solved the problem and suggest you patch it if possible:
while the session table is thread safe, the usage of ssl_decodePacket isn't thread safe, even if the the same session is always treated by the same thread:
note that whenever ssl_decodePacket is called, if syn is encountered the "CreateSession" method is invoked, which in turn calls RemoveStaleSessions. there, "old" sessions are removed, but no one makes sure that another thread is not currently using these "old" sessions. the bug is that currently used sessions are considered "old". this can be easily fixed by adding the following (one) line to the "GetSnifferSession" method in "sniffer.c" after the session is pointer is acquired and verified as non-null:

static SnifferSession* GetSnifferSession(...){
    ...
    while(session){ //obtaining the session according to ipInfo tcpInfo passed to the function
    }
    unlock(&SessionMutex);
    if(session){ //checking if a session was actually found
       .... 
       session->bornOn = time(NULL); ///******* new code ! *********// <--------------------
    }
}

this patch allows multithreaded use of ssl_decodePacket (assuming of course session-wise load balancing between the threads), and will not cause thread X to free a session thread Y is currently working on.
in addition, this also fixes a functionality problem (as I see it...), you don't really want to throw away a session SNIFFER_TIMEOUT seconds from arrival time, but only if the session was silent for SNIFFER_TIMEOUT.

what do you think?

regards,
Dan

just a note:
I know I am not using solely the sniffer application, and it's most possible the error is in my addition of the multithreading, I have looked at it hard before posting. If you feel this question is out of place in the forum I completely understand. just a reply of whether my understanding of "snifferServer" is correct would suffice if you are not able to elaborate...


thanks,
Dan

Hey all,
I monitor my own server and use the wolfSSL embedded ssl sniffer application using my server's private key. I register my server using ssl_setPrivateKey and do this only once (since i have only one server i monior). then, I call ssl_decodePacket for each packet. everything works just fine. recently, I tried to multithread the sniffer - I spawn some threads and call ssl_decodePacket from multiple threads. as far as i can see, the session table is thread safe, and it seems the application as a whole is built for it. however, I seem to crush when I ssl_decodePacket in a multithreaded fashion. I suspect it has to do with the fact that all the sessions to my server are using the same SnifferServer object, although, as I imagine, that has to be the case since I have only one server, accepting traffic on port 443.

I know SnifferServer is not being freed, this isn't the problem - I know it's quite general, but, any ideas?

sure, I can use the program with one thread, but I find the multithreaded version much faster to go over large data sets.

thanks,
Dan

Hey Chris,
thanks again for your quick reply and sorry for my delayed response. I agree with the logic you presented, however the fact remains that the value I received for *sslBytes was negative. I suspect that "reassemblyList->begin" value is too large, I will therefore go to check the values there and investigate whether I have made any changes to the reassembly process which could have cause this, or maybe there is a bug. I am inclined to think reassemblyList->begin is "wrong" since the rest of the values reflect the latest incoming frame, and no changes are made there, while reassemblyList->begin depends on more logic. I will keep you posted if I come up with any conclusions...

thanks again,
Dan

Hey Chris,
thanks for responding and sorry for my late reply.

Can you explain more about how and why you modified the value of "expected" in AdjustSequence()?  I think this might be what's throwing your sslBytes calculations off, thus causing the crash.

I didn't explain myself vividly enough. I am not modifying anything: I posted the values of the relevant variables as they were extracted from the dump (.core) file. Although the value of expected = session->srvExpected was extracted from the core file as being 1461, I specified that in order to truly reconstruct the path flow of the program, one should use expected = session->srvExpected = 4389. the reason for that being that the .core file gives the last "image" of the memory stack, and the value of session->srvExpected is modified (in the original code!) by adding to it the value of *sslBytes. since we are actually looking to trace back to the events that originally led to the value of sslBytes being negative, we should not use the values from core file effected by this undesired situation, but rather try to understand what were the values which lead to that situation. therefore, since session->srvExpected is modified AFTER sslBytes is obtained, the core file value for session->srvExpected is not relevant, we should try and conclude what was the value before hand (since it effects the flow of the program). due to the change being session->srvExpected += sslBytes, and sslBytes in the core file is -2928, I simply noted that one should use with session->srvExpected + 2928 in order to trace back what lead to sslBytes being negative.

to sum this: you can simply try and trace the code given the variable values I listed above with expected = session->srvExpected = 4389 and I am sure you will see the problem I encountered.

If you don't mind me asking, what kind of project are you working on?  If you'd rather not post this info on the forums, feel free to email me directly at support (at) wolfssl (dot) com.

I monitor my own server and prefer to do so passively (less intrusive to the application). the crushes I suffer occur rarely, but I imagine that since I use it continuously, even the less probable session values occur.

so, are you sure

*sslBytes -= newEnd - reassemblyList->begin;

is necessary? it appears to me that even when removing it should still work (in fact, I tried it and it does work on my various examples). However, I am concerned that maybe it is necessary and I will now mess up a different session scenario.


thanks for your attention,


Dan

Hey All,
I have noticed a rare crush in the sniffer Application. I debugged the Core file and reached a situation where the variable
sslBytes takes a negative value. Consequently, the following line in sniffer.c resulted in a crush:

in CheckPreRecord:
...

XMEMCPY(&ssl->buffers.inputBuffer.buffer[length],*sslFrame,*sslBytes)

it's clear why the code crushed here (sslBytes = -2928)
I started digging by looking at the values from the core file, following the entire path of the program, and got to the following line, where the I suspect the problem lies:
in sniffer.c, AdjustSequence(...):

if(reassemblyList){
   word32 newEnd = *expected + *sslBytes; //my calculations yielded newEnd = 4472
   if(newEnd > reassemblyList->begin){ //reassemblyList->begin = 1461 so this is true
      *sslBytes -= newEnd - reassemblyList->begin; //--> this yields sslBytes -= -2928
   }
}

why is the last line necessary? I couldn't understand, we already remove overlapping a few lines above...

for your convenience, here are the relevant values extracted from the core file:
packet = 0x7F696F4d28F6
ipInfo->length = 20
tcpInfo->length = 32
tcpInfo->sequence = 27041368
ipInfo->total = 299
length = 299
seqStart=session->srvSeqStart = 4225
expected = session->srvExpected = 1461, but you should use 4389 - see ** below
reassemblyList->begin = 1461
reassemblyList->end = 2868

i think these are all the values you need in order to trace the path of the program and reach teh same conclusion as I did...
**heads up:
in the function AdjustSequence, we modify the expected param by reference using sslBytes, which it helped compute. that means that in the core file we witness a value of $expected which is already influenced by the false calculation of sslBytes. since we add sslBytes (-2928) to expected after the calculation, I added 2928 to the value of $expected in the core file, which resulted in 4389. using this value, i was able to trace the entire program path to and get the value of sslBytes as it was seen in the core file.

if you need any more explanation of what I did in order to reach those conclusions, please let me know...


Dan

15

(2 replies, posted in wolfSSL)

Hey,
sry for the late reply.
I am using the sniffer application in order to monitor one of my servers for personal use. since I am using it rather constantly, I am encountering some of these errors.


thanks,
Dan

16

(2 replies, posted in wolfSSL)

Hey All,
I have been using the Sniffer Application over the past few months - I noticed that once in a very long time, the application crushes with a seg fault. I have set up a testing environment and by inspecting the core files I have discovered the following variables were uninitialized and most likely have caused the seg fault (I was unable to reproduce the crush after fixing those):
in src/Internal.c:
in function InitSSL - need to add:

ssl->keys.encryptSz = 0;
InitCipherSpecs(&(ssl->specs));

I suspect that after a long enough time, a new session is allocated with a "dirty" encryptSz and this causes a memory corruption followed by a seg fault later on.

second initialization required is interesting: the function "InitCipherSpecs" is actually supplied in the sniffer application but never used, I just added a call in the SSL object initialization method and it appears to have resolved the second seg fault i have encountered.

thanks in advance for considering these additions,

Dan

Thanks for your reply.
I saw the "rollover" part in the code, I agree that a connection of more than 4Gb would cause the error I witnessed. However I am pretty sure that wasn't my case (it happened quite quickly after I run the application, no time for 4Gb to accumulate).
If I understand correctly - the rollover handling that takes place only addresses the situation where the current packet's sequence has rolled over and therefore is much smaller than the *expected sequence number* - in which case "real" is adjusted. however, what if the following scenario takes place:
the expected sequence number is close to 2^32, a packet with that SN arrives and this packets "ends" after 2^32. I believe a packet of this sort will raise the assert call in CreateBuffer, or worse (in Release mode), will cause a seg fault due to the later execution  of memcpy with a negative size_t value of [end-*begin] (which is negative, as the packet starts at around 2^32 and ends not far from 0).

does this make sense to you?

I hope I understood the code correctly.

thanks for your attention,

regards,
DanC

[Update]:
I am not sure about:

if(added < 0){
    added = (0xFFFFFFFFU - *begin) + end + 1
}

do we need the +1, maybe we need more? or less? I suspect this is not accurate... you ideas will be much appreciated..


DanC

testing on my part shows that adding the mentioned code solves this particular seg fault which resulted in release mode (the assert wasn't active and thus a later memcpy used the *negative* $added value as param of type size_t to memcpy ---> seg fault).

1. i am still not sure of the +1 at the end of the expression there
2. while not yet sure, it might be that this fix alone is not complete, for now it appears to have solve this particular seg fault


DanC

Hey all,
I got a seg fault while working on the sniffer application. If I am not mistaking, in the "CreateBuffer" function in sniffer.c, the:

assert(begin <= end)

might be wrong... I believe I have encountered a packet (I went through the core file to check the values) which has a sequence number of about (2^32-11) and sslBytes for that packet was about 213... resulting that the packet ends "at" 202. I suggest to add:

if(added < 0){
    added = (0xFFFFFFFFU - *begin) + end + 1
}

instead of asserting...

I will test and let you know if it worked. I suspect however this little change might cause other issues if not supported throughout the code...

let me know what you think,


regards,
DanC

21

(0 replies, posted in wolfSSL)

Hey all,
could anyone please point me to where in sniffer application the validation of the encrypted handshake message is performed? I tempered a bit with the library and noticed that theoretically, even if the key derivation process fails (meaning, a wrong key is derived in the ProcessClientKeyExchange function), the sniffer application will still attempt on decrypting the session the session (unsuccessfully of course)...


thanks,
DanC

hey all,
I've been working with the sniffer application, and the following line baffles me:
(sniffer.c, AdjustSequence function):
~line 2090:

static int AdjustSequence(...)


if(real < *expected){
    int overlap = *expected - real;
    *sslFrame += overlap;
    *sslBytes -= overlap;
}

if(reassemblyList) {
    word32 newEnd = *expected + *sslBytes*
  [b] if(newEnd > reassemblyList->begin){

        /* remove bytes already on reassembly list */
        *sslBytes -= newEnd - reassemblyList->begin;
    }[/b]

to my understanding, after performing the overlap shifting in the beginning of the function, the sslFrame pointer points exactly to where NEW data begins and the sslBytes holds the number of NEW bytes (that is, the number of bytes in the packet which were not previously handled).
I am not sure to we have to perform the part in BOLD. it says, to what i understand - check whether the newly arrived packet ENDS after the last treated data BEGINS.

I think I lack the understanding to why reassemblyList->end is not exactly like *expected...

thanks in advance,

SheldonC

hey,
I have been using the sniffer program for some personal learning, I have encountered the following scenario:
the function AesDecrypt from ctaocrypt/aes.c was called with an aes (type AES*) param whose "rounds" member was zero,
meaning:
aes->rounds == 0
later in the code, it states:
if(r > 7)
       return;

afterwards we go on a for loop which will take place r times (breaking from the loop in the middle of the r-th time). in the loop, we raise rk (which is initialized to point to an array of size 60) by 8 and lower r by 1, per loop. if r was zero before entering the loop, we will effectively have r=0 for the first half of the first time in the loop, then, since r is of type word32 (=unsigned int), we will make r be (2^32)-1 after the r-- operation, thus resulting in a an almost infinite loop, during which rk will definitely exceed the bounds of the array it points to, which will result in a segmentation fault.

I am not sure whether the traffic i was testing is considered "correct", maybe I accidentally  stumbled upon some corrupted packets, but anyhow,
i think it could be wise to change the return condition before entering the loop to:
if(r > 7 || r == 0)
        return;

hope this helps someone,

SheldonC