I just found that Microsoft has published a use case about the way MySpace is using Service Broker on their service as the core message delivery system for the Service Dispatcher. We’re talking here 440 SQL Server instances and over 1000 databases. Quote from the use case:

Service Broker has enabled MySpace to perform foreign key management across its 440 database servers, activating and deactivating accounts for its millions of users, with one-touch asynchronous efficiency. MySpace also uses Service Broker administratively to distribute new stored procedures and other updates across all 440 database servers through the Service Dispatcher infrastructure.

That is pretty impressive. I knew about the MySpace SSB adoption since the days when I was with the Service Broker team. You probably all know my mantra I repeat all the time “don’t use fire and forget, is a bad message exchange pattern and there are scenarios when the database may be taken offline”? Guess how I found out those ‘scenarios’… Anyway, I’m really glad that they also made public some performance numbers. Until now I could only quote the 5000 message per second I can push in my own test test environment. Well, looks like MySpace has some beefier hardware:

Stelzmuller: “When we went to the lab we brought our own workloads to ensure the quality of the testing. We needed to see if Service Broker could handle loads of 4,000 messages per second. Our testing found it could handle more than 18,000 messages a second.”

Sometimes you are faced with applications that seem to take ages to start up. Usually they freeze for about 30-40 seconds and then all of the sudden they come to live. This happens for both native and managed application and it sometimes manifest as an IIS/ASP/ASP.Net AppPool starting up slow on the first request. The very first thing I always suspect is code signing verification. When a signed module is checked the certificate verification engine may consider that the Certificate Revocation List (CRL) it posses is obsolete and attempt to download a new one. For this it connects to the internet. The problem occurs when the connectivity is either slow, or blocked for some reason. By default the verification engine will time out after 15 seconds and resume with the old, obsolete, CRL it has. The timeout can occur several times, adding up to start up times of even minutes. This occurs completely outside of the control of the application being started, its modules are not even properly wired up in memory so there is no question of application code yet running.

The information on this subject is scarce to say the least. Luckily there is an TechNet article that describes not only the process occuring, but also the controlling parameters: Certificate Revocation and Status Checking. To fix the problem on computers with poor internet conectivity, registry settings have to be modified in the HKLM\SOFTWARE\Microsoft\Cryptography\OID\EncodingType 0\CertDllCreateCertificateChainEngine\Config key:

ChainUrlRetrievalTimeoutMilliseconds

This is each individual CRL check call timeout. If is 0 or not present the default value of 15 seconds is used. Change this timeout to a reasonable value like 200 milliseconds.

ChainRevAccumulativeUrlRetrievalTimeoutMilliseconds

This is the aggregate CRL retrieval timeout. If set to 0 or not present the default value of 20 seconds is used. Change this timeout to a value like 500 milliseconds.

With these two changes the code signing verification engine will timeout the CRL refresh operation in 500 milliseconds. If the connectivity to the certificate authority site is bad, this will dramatically increase the application start up times for code signed applications.

I am following the 37signals blog ever since I kinda randomly stumbled upon their Getting Real book. If you never heard about them, definitely check out the book, is a very common sense approach to managing product development in the age of Internets. On this post http://www.37signals.com/svn/posts/1798-jasons-talk-at-big-omaha-2009 I was really touched by one remark: Inspiration is perishable. The ideas you have can linger in your head for a long time, but the inspiration for it fades quickly. So don’t postpone it, by the time you get to it you’ll only deliver a pale image of the original idea. Do it when you’re pumped up and thrilled by it.

I reckon I’m a procrastinator deLuxe, but I have to agree. I know the difference between working at 2 am. and not feeling a bit tired when I’m excited about my work on one hand, and the damp feeling of exhaustion that drags you to watch some stupid TV show at 6 pm because I’m bored with the current project on the other hand.