Last month I have noticed that my blog is not showing up with Google search results. Using Google Webmaster Tools I have discovered multiple 403 (Access Forbidden) errors thrown by web server on Googlebot attempts to index website.
After small investigation I have figured out that my hosting provider (Hostingfuse) has blocked Googlebot crawler (without notifying me, thanks guys) due to website performance issues.
ColdFusion instance on the server was locking up and restarting every 5 minutes upon website indexing by Googlebot. Pages generation was extremely slow taking from 20 seconds up to 20 minutes to complete.
Reducing Googlebot crawling rate and pages caching did not help as server loading was still high, so after playing a bit with scripts and SQL queries I have noticed that MySQL database response time was ridiculously slow. By tweaking local MySQL instance, tables' structure and queries I found a solution in the end: switch tables type from InnoDB to MyISAM.
Now database taking less space on hosting server (MySQL official docs
says: InnoDB tables require a lot more disk space than MyISAM tables) and queries are fast as they should be.
So, consider database type for each project separately, not sticking to your past experience, even if it was positive.
