It seems I did something very stupid recently - I actually turned on a feature in Gmail that was listed under [labs]. The feature in question is called "smart labels" and as a feature it does worse than fail - it has convinced me that I need to move to a different mail provider.

The newly created labels were hidden under the [More] button which makes them invisible (I just now found the way to force the system to always show a label). Once messages are tagged by this add-on they disappear from the inbox. A whole selection of important messages have been tagged as "Bulk". Many of these so called "bulk" messages should be unique to me (I seriously doubt my financial institutions are sending my account summary messages to anyone other than me). So far this "mis-tagging" has cost me at least $200 due to a second chance offer on eBay that was flagged as "Bulk" and thus missed. I am not done adding up the damages yet, time will tell how much this little mess will cost me.

Maybe this is a wake up call for me to shift my accounts away from Google as my primary service provider - "do not be evil" seems to be out of vogue at the minute.

My experiences with VPS.net

I have used a handful of hosting providers over the past 10 years. Most have been average - a couple have been great and a couple have been awful.

Recently I have been using VPS.net as my primary hosting provider. Using a Linux VPS is not for everyone because it places the entire burden (and it is a burden) of managing the system on you the owner. However I have had enough training and experience in the Linux ecosystem that I felt confident I could keep up with what was required. (I have been actively using Linux since 1995 and I am not referring to KDE or Gnome or even X but rather to the shell)

So what can you expect from VPS.net as a hosting provider - there are a lot of people recommending them. There are a couple of good reasons for that - they offer a nice bundle of services and quite a few discounts of various kinds. Under regular usage they do perform fairly well although they appear have a few more outages than some of the more established providers. The proof of any service provider however is in how they handle problems. Do they really care about their customers or are they only in business to make money. (This is an important thing to keep in mind when reading recommendations for hosting providers as well.)

Here is my experience (and why I am not able to recommend VPS.net to anyone even though I want to).
I have been a customer for a while (a little over a year) and while I am not a particularly large customer I do manage over 30 domains which together often pull over one thousand visitors per day (in the past 30 days I have seen over 30k unique visitors). I also want to say up front that I am not the most technically competent hosting provider. I am pretty good at what I do, I rarely find a problem I am not able to trace and eventually fix, however being a systems administrator is not my only hat so it can take me a while to track down the cause of a problem and I know there are mistakes that I make and things I overlook. If anyone reading this post has suggestions about things I could have done differently please let me know as I am always trying to improve.

About a month and a half ago my primary production server which hosts about 30 domains crashed. Well, not quite crashed - the file system switched to Read Only mode due to some errors and several files were corrupted. I had to reboot to single user mode and run fsck before I could continue using the server. That took about 45 minutes during which time all the websites hosted on this server were not available. After checking the logs I saw some disk errors so I contacted support and asked them to check their logs to see if they could suggest the cause of the problem (if it is on my end I want to fix it). They reported back that there were no errors which is puzzling - first of all I am running a VPS that is running on Xen - the disk in question is a SAN - between these two points you will notice that the system does not run on a literal disk but instead disk access is virtualized so any disk errors will be related to the infrastructure. These errors then will be occurring on a piece that is outside of my control and in theory should be logging errors inside Xen or the SAN (or probably both).

Within two days I had another crash identical to the first - by this point I had tracked the problem down to a specific server application and website application - MySQL was initiating the crash, and the queries were originating at a copy of Simple Machines Forum. This still didn't explain the I/O errors but at least I had something to go on. So although this server instance is not "full" I paid for another two nodes - setup a brand new server instance and moved that one site to the new server. It is significant to note that since this move the server that was originally affected has not had any problems so the problem is most definitely related to that application instance.

Now with a new server I started working with support more aggressively. I was seeing the system enter R/O mode as often as every other day (once even 2 times in one day). When I would open a ticket support would take a look - tweak something (I suppose) and tell me it should be fixed - often by the time the intermittent problem returned the ticket would be old enough that they would have it closed so I ended up opening 3 tickets for the same problem (apparently you cannot re-open a ticket). When I would open a new ticket I would reference the old ticket but it seems that did not do any good because I did have to re-explain details previously covered several times. Eventually they admitted there was a problem and "moved me to a quieter part of the network". Since then there have been two more crashes but it has been two weeks now since the last problem so I suspect things are stabilized. In the mean time I have had a production server up and down repeatedly, I have had to refund my customer his hosting fees because the service I was able to provide him was poor. And finally to add insult to injury I missed a sale on new nodes. They did not post a close date on the sale, but with all the problems I was having I did not want to buy nodes until I saw that they could fix their problem.

Everyone makes mistakes, and everyone deserves a second chance. My Technical experience was awful but my customer service experience was not necessarily over so I decided to contact management at VPS.net and see what they had to say. Their response was interesting
1.     I missed the sale end date (which date was kept a secret) so that was that - no sale for me
2.     The outage I experienced that was because of a SAN problem that they knew about but denied (which information I learned after I read in the forums) was my problem - I do not qualify for a refund or any kind of discount (I did get a weak apology but they are unwilling to put money where their mouth is)
3.     They are replacing some of their software (with OnApp) which will "reduce problems in the future" (however my complaint is not so much with the technical problems or even with the financial but almost entirely with the way they have treated me - claiming known issues were my fault - dragging their feet on any kind of fix - making me open more and more tickets)
I would have to say that while they are eager to acquire new customers, they have little interest in taking care of existing customers. Customer service is 90% of the customer experience - claiming that things will improve in the future will not prevent me from going out of business today because of their mess, and they have left me feeling very much that they do not care if they lose me as a customer - they will just get another one that is not so demanding.

In conclusion I still like some things about VPS.net and I dislike some things. Their customer service leaves a lot to be desired and this is a concern because they are not addressing the issue. They say they are addressing the issue – by hiring more people – but I used to work in customer service (actually because I own and operate a business – I am very much in the customer service business right now) so I speak from experience when I say that hiring new people impacts response time – it does not change the service offered. Service is a personal thing – implemented one rep and one customer at a time.

So I find myself open to the possibility of another provider – I gave my current provider multiple opportunities to either work with me or provide assurances that they were working towards being able to and the assurances I received were not reassuring. I do need VPS level access although I could make do without cloud scalability it is a nice feature and it is hard to go down the feature ladder. I have heard very good things about RackSpace and have even worked on a server hosted with them a couple of times – they are next on my list to check out. I am actively following the new OpenStack initiative and on top of that two of my friends even work for them so this is probably well past due.

Why I no longer use 1and1

My first (personal) hosting package was provided by 1and1. I used them for simple site hosting and even recommended them to friends an clients for many years. They provide shared hosting and most of my experience with them was satisfactory.

I had an experience however which changed my views on them forever (and have since learned that this problem is common to many shared hosting providers).

I started having a problem - code was being injected into my web pages to run drive by attacks - basically to use my sites to attack my visitors computers. This was accomplished by inserting a <script> block just before the </html> tag. I cleaned up the system a couple of times then decided to make the time to fix the problem.

I figured the problem fit into one of 4 categories. 1) a vulnerability in a piece of application code 2) someone had one of my account passwords 3) my personal computer was vulnerable 4) the server was vulnerable

so I started out to determine which of these caused the problem.

First to eliminate possibility #1 (the most likely in my estimation) I removed all server side scripted content from every website hosted on this account (php and perl in my case). I had to move a couple of sites to another provider to do this but when I was done 100% of my sites on this account were running plain old html (all files ending html, htm, css, js). The only file other than those was a single sh file that I used to scan for and list the inclusion of the attack code which I placed in a folder not visible to the web. No php, no perl, no ssi. And what did I find - I found that my code was still being modified by an attacker.

Next up - I changed all my passwords (1&1 control panel, ftp, shell, ssh-keys, mail) and did not share any of the new passwords with anyone. I then rechecked for any scripts - only the one mentioned above (a single shell script was present) so I cleaned up the files (again) and set the file permissions to read only (444). Another week goes by and the files have been modified again.

Next the scary #3 I checked over my computer very thoroughly - while this option seemed least likely to me (I had other hosting accounts through other providers that were not being attacked in this way which makes this scenario unlikely) I figured it was a good time to review my local security footprint. I worked through the firewall rules, ran a full scan using updated antivirus definitions, ran a full scan using updated anti-malware definitions, and ran a rootkit check (all from a bootable CD created on another system to be safe). Nothing turned up on this side (as expected). (Note that this does not entirely rule out the possibility that my system was compromised).

To the best of my ability to tell there was a vulnerability on the server that was allowing a 3rd party to access and over-write my files at will. As this attack was not being initiated through a vector over which I had control there was nothing I could do to fix the problem from my end and technical support did not believe they had a problem.

At this point I switched providers and have never looked back. To date (3 years later) I have not had any further problems of this kind.

Discounted prices on books and software

So I have been poking around the internet for years and during that time I have kept my eye out for anything of interest related to my other principal information source - Books.

Over the years I have of course bumped into many resources some more useful than others. I am not able to find a good catalog of useful resources online so I have decided it is time to create my own post here.

I am skipping the obvious like Amazon.com and resources that I have found less than useful.

• Addall simply the best in price comparison shopping for books
• LibraryThing.com book cataloging (I have not been able to find a better product although there are a half dozen fairly good competitors. I have my entire library cataloged and am working on improving my meta tags)
• Smarter Comics Business books in comic form - some eBooks for free

Always of interest however are discounts on the purchase of books.

Digital Books

• Gutenberg (out of copyright works with a couple copyleft and similar)
• Calvin College Ethereal Library Classical Christian titles
• Librivox ('gutenberg read aloud' or 'audible for free')

Tech eBooks (I check the daily or weekly deals at the following sites)

• CiscoPress
• PacktPub
• Manning
• O'Reilly
• InformIt
• QUE
• Apress
• FT Press
• Pearson IT Certification
• Peachpit

Software (ok not all books but still interesting)

• BitsDuJour desktop software - occasionally have a particularly good deal on a unitility such as Directory Opus

Hardware (There are a handful of options but the only one I still find myself going back to is)

• TechBargains

MySQL problems again and again

Another MySQL limitation. This time it is the lack of support for Full Outer Joins. I was really hoping to avoid using a UNION as in my opinion it makes the SQL much harder to follow. So I have to make up the difference with an extended comment block so the code remains maintainable. The workaround isn't too complicated but I strongly believe that the tools I use should be making my life easier, not making me spend a lot of extra time creating workarounds. The time savings issue alone will probably soon make me switch permanently off MySQL as this is not the first time I have had problems with MySQL not supporting the full ANSI standard. (actually the frequency appears to be increasing logarithmically as my projects increase in complexity).

I keep banging my head against the limitations of MySQL. I know I should probably switch to PostgreSQL as every time I find a hard limit in MySQL (so far) it would be solved by moving over. Guess it is time to start studying. The recent aquisition of MySQL by Oracle is another factor that is making PostgreSQL look enticing.

Basic Hosting Information for Everyone who wants a website

Two posts back I covered the levels of hosting, however that didn't really meet the needs of most of my clients. So I went back to the drawing board and created an article that attempts to cover all the technical details you need for your first website in an accessible format.

Read it here.

Drupal Migration

Just moved a drupal instance from a local dev server to a public production server. Was getting a 500 error but a little research showed that I needed to set a value in the .htaccess file - specifically

RewriteBase /

This diagram displays the tiers of the system required for a website. When you have a website it is running on this stack.

First to define the tiers
E) Network and Power – physical network connection and power to run a server
D) Server hardware – the actually physical server (like your desktop pc)
C) Operating system – usually Linux or Windows – allows the other software to run on the hardware
B) Server Applications – on Linux (LAMP) this usually includes Apache, MySql, and PHP. These pieces are the final level needed to run the website itself.
A) Website / Applications – this include the website itself and required applications like Joomla or Wordpress

Now let us take a look at some various hosting options.

1) Full Hosting. In this case you provide A-E
2) Co-location. In this case you buy the E from a provider. You then buy a Physical Server and they install it at their location. If something goes wrong with the network they fix it. A-D is your responsibility. You get the server all to yourself.
3) Dedicated Server. In this case you are leasing a server from them (D-E). If something goes wrong with the hardware, they fix it. You get the Physical Server all to yourself. A-C is your responsibility.
4) Virtual Private Server (VPS). In this case They have a server and you are buying a piece of it. They set the operating system. Everything above this point is your responsibility. Physical Server is shared – usually between 20 and 100 other users. Often they will also have a standard set of Server Applications that cannot be changed. Usually you can install applications. So you are responsible for A and maybe part of B. They provide B-E.
5) Shared Hosting. In this case you are leasing space on a server. They set almost everything. Sometimes you get terminal (SSH) access but usually not. You cannot install any applications. You share space on a physical server with usually 100-500 other users. You are responsible for A. They provide B-E
6) Application hosting. In this case the application is installed for you. You cannot create any new pages. You can use the specific application. You are responsible for the data. They provide A-E.

It is very important to note that each level is available through managed hosting and most are available as unmanaged hosting. What is the difference? Managed means the service provider takes care of administrative tasks. Unmanaged means it is up to you to take care of these issues. For example at the server level. If it is unmanaged then if a fan fails it is up to you to get it fixed. If it is managed – they will take care of fixing it.

Extras
1) Hardware firewall - position (Z) - Some providers offer a dedicated firewall solution. Often a Cisco or Juniper device.
2) Server Firewall - position X
3) Application Firewall - position X/W (depending on the solution)
4) Redundancy - often located at position (Y) this allows the site to keep working even if a server fails. Not available in most packages - instead you must wait for server to be replaced and restored from backup.
5) Scalability – position varies – if your needs expand can you expand the services to match.
6) Backup - may be located at positions (Y - system image), (X - server instance image (if you have 100% of server doesn’t differ from Y)), (W - website backup), (V - website data backup) depending on what we are backing up and how. Most solutions offer limited backup options.

Very few people really desire to manage the whole thing or know how to do so. If that is you – then you will definitely want to choose Full Hosting as it gives you the maximum flexibility. This is where many medium and large corporations operate.

For the rest of us – we need to pick one of the other options. Which option you choose should primarily be dictated by what you need to get done. As you work your way through the options there are two things that change.

First is how much of the server is yours – as you can see with shared hosting you get around 0.5% of the server. On a VPS you may get 5%. On a dedicated server you get 100%. There are solutions available that allow you to use multiple servers. So how much of the server is yours.

The second thing that changes is your responsibility. As you get more and more of the server – you usually also get more of the responsibility for keeping it working. Increasing to a bigger package doesn’t just give you more resources – it also gives you more options and responsibility. If you want to scale up to more resources without additional responsibility then you must go with a managed solution. This is an add-on to many packages where the hosting provider takes care of some of the administrative burden for tiers that they would not normally cover for that package. For example on a dedicated server – you are in charge of the operating system, the server applications and the website. If you pay for managed hosting they will take care of some or all of these levels for you.

Magento related products issue

I have been working off and on with a design shop getting a new Magento multi-store instance live. The latest issue is somewhat interesting. We are currently showing a 'Related Products' column and for one of the store instances there were no checkboxes available to select a related product - after a little bit of research I found that if the item is out of stock, the checkbox will not be displayed.
For Reference..

Twitter profile pull

One of my friends wanted to pull profile data from twitter and asked for some basic code. I am posting the mockup as it might prove interesting to someone else.

---

<?PHP
   $output = $xml_content = '';
   $tw_name = 'jeremiahstover';
   $xml_content = simplexml_load_file("http://twitter.com/users/show.xml?screen_name=$tw_name");
   $xml_content2 = simplexml_load_file("http://twitter.com/statuses/friends/$tw_name.xml");

$output .= "
   <html>
      <head>
         <style>
            .tw_profile{
               width:200px;
               border:1px solid black;
               overflow:hidden;
            }            
         </style>
      </head>
      <body>
         <div class='tw_profile'>
            <a href='". $xml_content->url ."'>
               <div class='avatar'>
                  <img src='".$xml_content->profile_image_url."'>
               </div>
               <div class='avatar_title'>".$xml_content->name."</div>
            </a>
            <div class='description'>".
               $xml_content->description   ."
            </div>
            <div class='friends'>
               <strong>Friends:</strong><br>
               <i>  ";

               foreach($xml_content2 as $friend){
                  $name = $friend->name;
                  $url = $friend->url;
                  $output .= "<a href='$url'>$name</a><br>";
               }

             $output .= "
             </i>
          </div>
       </div>
    </body>
</html>
";

echo $output;

?>

---

Local Instance of Magento

I have been playing with a local instance of Magento - working on a custom template. When running on Windows (through WAMP). To get login working there must be a dot in the base site name - thus http://localhost/magento will not work - instead install to http://127.0.0.1/megento and you have a dot in the name. This issue is related to the way that cookies are handled internally and will not present any difficulty on a publicly hosted site.

Hack attack

I have been fighting an attack on the integrity of my websites for a little over a year now. At first there was no information online, but as the attack has continued I have found a few other people that suffered from the same attack. I finally decided it was time to put in the effort to track the issue down and correct it.

The attack was a code insertion in the footer of my pages. The attack script apparent searches for </body> and inserts a little piece of javascript right before it. The javascript loads a file located on one of a series of servers in china (domainname.cn).

At this point I have a shell script that I am running sporadically - searching all files on my server for the telltale indicator. It compiles a list of the files - which I then send through a second shell script which removes the attack code.

It appears the attack originated with an old instance of phpBB - which I have removed. Once the attack was successful a series of backdoor scripts were uploaded - all of which accept a posted file and then eval it. (PHP local execution). I have a second script that is looking for all instances of eval and listing them in another file - this one has to be checked by hand at this point as I have not yet selected a unique identifier for the attacking code. Once this part is cleaned up I should be clear (finally), hopefully for a while.

I am pondering things I can do in a shared hosting environment that would make this kind of attack more difficult. The best I have thought of to date is to setup a pair of scripts to set my websites to 'read only' and to 'read write'. Each time I make an update I could then unlock the files, and when I was done I could lock it back down. Without paying for a VPS, this may be the best solution. Another option would be to run a daily cron job that scans all folders for files changed in the last 24 hours. Have it compile a list and email that list to me. This way I would be notified any time a file changes. I would have to tweak the settings as there are a couple of applications that I am running that make use of temporary files. Temp files would have to be scanned to make sure they were less than x days old (where x is about 2) and deleted once they are older than that. All to prevent the temp folders from being a useful attack vector.

I suspect there is a better approach to the problem than either I have specified - even thought I am on a shared hosting account. I will give it some more thought as I have time.

Wordpress template excerpt

I needed to add excerpt functionality to a custom WordPress theme and found the following article to be just the ticket.

Replacing Wordpress content with an excerpt without editing the theme files

Magento multi store granular permissions

Magento eCommerce allows granular permissions in there community edition. Granular permissions are accessed by creating a role and assigning users to roles. However the community editions granular permissions apply to all stores multi-store install. For granular permissions per store instance on a single install you need to purchase Enterprise which costs $8,900+ per year.

PHP and type casting

I was going through some of my notes and found an issue that gave me some trouble. About a year ago I was working on a custom shopping cart, and ran into trouble with PHP's type casting. I was multiplying a float (price) by an int (quantity) and PHP was converting the float to an int by dropping the decimal part of the number before doing the multiplication. Obviously this was causing trouble with the total price charged. The fix was to force the int to a float before the multiplication.
$lineTotal = (float) $price * $quantity;

Magento eCommerce

One of my clients is setting up a multi-store instance of Magento - and wanted to change each stores home page. I have not used Magento in a multi-store instance before so I had to look up the details.

Ok - here is what you want to know - I updated one of the stores - the required steps as follows
1) create a new page in the cms (named something like store-home)
2) If you have not already, create the new store
3) in admin open menu System->configuration
4) select box at top left - choose the entry 'english' under 'Maria's Botique'
5) left menu - choose 'web'
6) on right choose 'default pages'
7) modify the 'cms home page' by choosing the custom page we created in step 1
8) done

My reference material can be found here
http://www.magentocommerce.com/knowledge-base/entry/overview-how-multiple-websites-stores-work

SugarCRM

One of my clients is implementing an instance of SugarCRM - the recently setup several webToLead forms and wanted to require a set of fields other than the default. After poking around a little (on the documentation) I took a look at the page source. The solution proved quite simple.

<input id="req_id" name="req_id" value="first_name; last_name;webtolead_email1;" type="hidden">

As you can see - adding new id entries to the value of this field makes those fields required.
P.S. name and id appear to be set the same on every field - but the javascript appears to be using the id for verification.

Facebook advertising

One of my clients has decided to run a simple advertisement on facebook - the stats so far are interesting. We choose to go with pay per impressions to start with. The ad links through to a page on facebook, so there is a 'fan' link at the bottom of each ad.

Currently we are seeing
a) 1000 impressions per actions
b) 1.8 actions per fan
c) 8 fans per person who follows one of the links on the facebook page through to our website
d) 2 people on our website per 1 who fills out the form for the free promotional item.

This will bear watching to see how things develop as we attempt to tweak the settings

(UPDATE) We tried an ad the directed people to the website form directly instead of going through a facebook page - all that seemed to do was drop us straight down to step c - which means fewer fans but the same number of website visits per advertising $.

Development challange

I have recently been working on Joomla - running among other things an instance of Mosets tree 2.1. I have been working on a custom import from an external database. While you can get Mosets to rebuild its hierarchy using
http://www.yourdomain.com/administrator/index.php?option=com_mtree&task=rebuild_tree
this will not always manipulate your data into a form that Mosets can use.
I ran into trouble until I found that the root element must have an id of 0 and it must have a parent of -1. Any id other than 0 will not fail catastrophically, however it will not work properly either (I was seeing around 5% of the elements in the table showing up).

Desktop utility

I don't find myself using many windows add-ons - but this one is actually proving useful.
Stardock Fences