Still importing logs

Well, it seems that SmarterStats still hasn’t fixed their biggest bug… when you import a bunch of log files, the service will mysteriously stop importing.

For example, I have log files starting from January of 2008.  When it imports, sometimes it will get to the middle of June, sometimes the middle of December, and then it just stops importing.  To reimport, it deletes all of the processed logs and starts from January 2008 again… it’s a vicious cycle.

I’m trying a solution where I move the log files that it has processed to an archive, and also move the distilled “processed” logs to another directory.  When it compiles a new set of stats, I will merge it with the existing stats.

I also changed the settings to not pause.  By default, the service pauses every 15 minutes to “give your hard drive a rest”.  WTF is this crap?  This is supposed to be enterprise-level software.  When importing logs, I never see more than 5% CPU usage, and hard drive activity is nothing unusual, either.  Why in the hell does it take so long to import logs when resources obviously aren’t being utilized to their fullest potential?  Regardless, I set the “timeout” from 15 minutes to 999 minutes, and we’ll see if that resolves the problem.

It’s very tedious.  This problem has been around in SmarterStats for years.  If you troll their forums, most of the posts are about this mysterious behaviour.  Like I said in my previous post, I have a love/hate relationship with SmarterStats.  Once the logs are imported, it is very powerful… it’s just getting to the point where everything is imported.

Alan is a web architect, stand-up comedian, and your friendly neighborhood Grammar Nazi. You can stalk him on the Interwebs via Google+, Facebook and follow his ass on Twitter @ocmodshop.