phone and contacts app freezes becomes slow and scrolling jittery iPhone 4

My setup:
iPhone 4 with iOS 4.2.1 (8C148)
I dont have 3G on my CIM (provider BASE.BE), I can only use WiFi to connect to internet. 
I sync my gmail contact (mail and calendar are turned off) through the iPhone built in Exchange account connector.
Fetch new data is off, push is off, fetch set to manually, advanced schedule set to manual.   
I only turn WiFi on once a day, when Im home, sync contacts and then I turn WiFi off again.
Spotlight search is turned off.
Device runs fine, battery life is great (much better than my iPhone 3Gs), I have about 2700 contacts, no duplicates, updates in both directions, everything working perfectly.
 
Then without any warning, or even without any use of the phone, this happens: 
1) The phone and contacts app becomes unresponsive for a few seconds, remains very slow, scrolling becomes jittery or freezes.  When the symptom occurs both apps have this issue.  I assume because both use contacts.  Both apps dont crash, they just get very slow as if they were trying to do something heavy. This does not happen with any particular event, like getting a call or text message. This can happen even at night without any missed calls or text messages send or received.  The symptom does not affect any other app.  For example when I open the iPod app, scrolling though the long list of music is fast, without any freezing or jittery behavior, while the Phone and Contacts app is still very slow.  The problem does not go away by itself.  Hard rebooting the phone does not help.  I assume the problem is not related to any purchased apps because this can happen when none of them are running.
2) The phone gets really hot.
3) The battery drains very fast.  I can go from 100% to 0% in under 30 minutes.
4) CPU usage (use System App to measure) goes from < 20% to >75% or more.
I notice the symptom mostly when in the 1) phone app, or 2) I feel the phone heat up or 3) in the morning when the phone battery is almost empty when it was full the night before.
 
I can fix the problem by 1) turning on WiFi, opening the contacts app.  Then I see the circular indicator in the top left hand corner turning round.  This takes just a few seconds, to sync 1 or 2 contacts en when the indicator disappears, the problem is gone.  Scrolling is fast again.  Problem fixed.  It can take a day or two and the problem comes back.  Do sync trick again and problem gone again, and so on...
I can also fix the issue by 2) deleting my gmail exchange account, putting it back and syncing everything back.  But like with solution 1) the problem will just returns after a few days.
I have even tried a full factory reset and manually putting all my apps back.  But again, when I import my gmail contacts, wait a few days and problem returns. 
Ive been having this issue for a few months now and have done quit some research on the subject.  Im pretty sure I also had the issue on iOS 4.1 and possibly also 4.0.  I did not have the issue in iOS3.  I had the same problem on my iPhone 3Gs which Apple replaced.  I thought the battery was dying.  Now I bought an iPhone 4, same problem. 
I purchased many system monitor apps to find out what process was causing the problem.  The problem is that none of the apps I found report CPU or memory consumption per individual process.  So I only know CPU goes from below 20 to 75% and more, but I cant pinpoint which app or process is doing the extra consumption.  But it has to be one the standard built in iOS processes.  I can see when what process started.  And when the issue happens, I can hard reset and dont launch anything, and the problem is still there.
When iPhone apps crash and you connect your phone via iTunes the crash reports are put on your computer.  I found these logs but because the contact and phone app dont crash I have found nothing interesting there. 
I have not found a way to trigger the problem on purpose.  I tried adding, changing deleting contact on my iPhone, none of these actions trigger the issue.  The problem just appears at random.  
I have also not found a way NOT to have the issue.  Ive tried not changing any contacts for a few days, but the problem still creeps up. 
I have read dozens of threads on the internet and only this one comes close:
http://www.google.com/support/forum/p/Google%20Mobile/thread?tid=62b2355d61d8e11c&hl=en
I also found this interesting:
http://discussions.apple.com/thread.jspa?threadID=2567976
<Edited by Host>
n/a iOS 4 iOS 4.2.1 (8C148)