Drupal

Get Insights from Google about your Drupal Website using Webmaster Tools

Webmaster Tools, according to Google, is a "free service offered by Google that helps you monitor and maintain your site's presence in Google Search results" and "can help you understand how Google views your site and optimize its performance in search results."  It is an invaluable tool that you should be using.

Some of the things you can do with Webmaster Tools:

  • receive messages from Google about issues it has found with your site (including crawling and security issues)
  • be sure that Google can properly view all your content
  • understand which search querries are leading viewers to your site
  • see which websites are linking to your site (backlinks)
  • confirm that Google is reading your sitemaps
  • see how Google views your semantic microdata
  • find out recommended HTML improvements
  • learn if you have duplicate or short meta descriptions and page titles
  • see how fast your web pages are loading

This information covers the core of modern SEO practices.  But to view it and use it, first, you need to verify your site ownership with Google to begin using Webmaster Tools.  Which we hve covered in a previous blog post - Verify your Drupal Site!.  Do that first, then wait a few days for some data to accumulate.  Then come back here and we'll walk through a few of the most important Webmaster Tools reports.

First, let's look at the home page.  You should see all the sites you have added and verified.  And hopefully a "No new messages or recent critical issues." message for each.

Google Webmaster Tools home page

Click on your website name link to bring up the dashboard for your site.  If you have any messages, check those first.  It could be that Google can't index your Drupal site, or worse, that it found malware or a security issue.  Take care of those first- a security issue could get you completely blacklisted from Google!

Our example site below shows no messages or site errors that Google tracks.  There are some 404 errors shown under URL Errors.  You can learn more about those errors so you know what to fix under Crawl >> Crawl Errors.

You want to see those pretty little green checkmarks.  A upward trend on your Search Queries is also a good thing.

There's too much to cover completely here, but you should definitely check :

  1. Search Traffic >> Search Queries.  What terms are Google users searching for that leads them to your site.  There is also data on your click-through rate and the average position you show up on search results (10 per page).  Are these the terms that you want to be found for?  Are people seeing you in search results, but not clicking?  Here at FireRoad we once wrote a blog post about configuring a certain type of complicated large computer monitor we were using.  We get tons of traffic to that post, but it really isn't useful traffic since we don't sell or service those monitors.
     
  2. Search Traffic >> Links to Your Site.  Which sites link to your site?  Lots of quality links (that is, from sites that Google thinks highly of) helps your SEO.  it shows those sites respect your site and content enough to link to it.  Lots of un-quality links (that is, from Estonian link farms) can really hurt your SEO.  If you've been using offshore "$100 gets you on the front page of Google" link building services ... stop.  That might have worked years ago, but now it does the opposite.  You can also disavow any links that you don't want and request Google not consider them.
     
  3. Search Traffic >> Mobile Usability. Having a mobile-friendly website is critical now.  Google will tell you if it thinks you have issues with mobile usability.
     
  4. Search Appearance >> Structured Data.  If you are using semantic microdata on your site to better tell Google what your content means, it will show up here, along with any errors.  We will have a guide to using microdata on Drupal websites out soon.  You can also use Search Appearance >> Data Highlighter to manually identify it.
     
  5. Search Appearance >> HTML Improvements.  It only shows a few suggestions, but on important issues it may find on your meta descriptions, page titles and non-indexable content.
     
  6. Google Index >> Index Status.  Make sure all your pages are being indexed.
     
  7. Google Index >> Content Keywords.  What does Google think are the most relevant keywords on your website?  Are they want you want?  If not, change your content.
     
  8. Crawl >> Sitemaps.  Do you have a XML Sitemap on your website?  It helps Google understand your site structure.  It should show up here, and make sure all your pages are included.  If not, see our post on Setting up an XML Sitemap in Drupal.
     
  9. Other Resources >> PageSpeed Insights.  How fast does your site load?  The faster it loads, the better you will rank.

With very little work, you will be up and running with Google Webmaster Tools and using its free tools to better understand and improve your Drupal website.

Setting up Goals in Google Analytics for your Drupal site

"Setting goals is the first step in turning the invisible into the visible."
        - Tony Robbins

Quick - what percentage of all the visitors to your website use your Contact Us page to request more information?  On your "Download our Guide to Norwegian Cheeses" landing page, how many people give you their imformation and submit the form, and how many aren't convinced and decide to get their Norway cheese info somewhere else?  Are those numbers going up ... or going down?

You can't tell how well your page design and marketing efforts are working if you aren't measuring goals and conversions.  Fortunately for Drupal sites, with a little help from Google it's easy to do.

Let's start with your Contact Us page, and assume you are using the core Contact module.  If not, first set one up.  You will also need to set up Google Analytics and the related Drupal module if not already installed.  Once those are working, head over to Google Analytics, click on Access Google Analytics on the upper right side, select your website and All Web Site Data to get to your home Analytics page.  Look under Conversions >> Goals >> Overview to see all your configured goals ... which at the moment is likely none.

First step in setting up goals in Google Analytics

So let's set some up by clicking on the Set up goals button.  You should see this:

Click on +New Goal.  Google Analytics provides many templates for configuring your goal - choose Contact us and then Next Step.  You should then see this:

Step 2 of configuring a goal in Google Analytics

It is asking you to configure how you will consider this goal met.  The default type, and only one that makes sense for our Contact Us page is Destination, meaning that Google Analytics will consider the goal met when the user visits a certain page.  Specifically we need a thank-you page - a page that is only reached after the user submits the Contact Us form.  We can't use the URL of the Contact Us page itself - that only tells us that the user viewed that page, not that she submitted it.  After she clicks the Send Message button, she should then be shown a new page that thanks her for sending the message, and perhaps has some additional sales and marketing info on it.  But the important thing is that it is a different page, so we can track it.

Which is now a problem for us, as the Drupal core Contact module doesn't have the option to specify a redirect page that the site viewer is shown after submitting the Contact Us form.  When a user submits the form, she simply sees a standard green Drupal status message and stays on the Contact Us page, like this:

Not an especially good User Experience, and more importantly, it doesn't give us what we need to trigger the goal.  Adding that important functionality has been discussed for many years, and there is work going on now to put it into Drupal 8 ... but what do we do for all our Drupal 8 sites now?  We could use the Webform module to create our Contact Us page, which does have the redirect page option as well as many other additional features, but at the cost of highercomplexity and effort to implement.

Fortunately there is a perfect module that adds just the functionality we need - Contact Plus.  Install in your preferred manner.  Now, when we edit our sitewide contact form (Structure » Contact form >> Edit), we see a new configuration field for Redirect path.  

The default is to redirect to the front page, but that isn't useful.  Instead, create a new Basic Page that thanks the site user for submitting her info, promising them to get back to her shortly, and any other marketing messages you may want.  Set the URL to something like http://yoursitename.com/thank-you-for-contacting-us.  Put that entire URL into the Redirect path field (the module can't handle a relative path unfortunately) and save.

We now have what we need to finish configuring our goal in Google Analytics.  Go back to that browser tab, make sure Destination is selected and click Next step.

Enter just the relative path of your thank-you URL (the part after the site name) and click Create Goal. You can leave the other advanced options set to off. If you use a social tracking system that adds more characters to the end of all your website URLs (like ?JK8uejshl), then change Equals to to Begins with or your goal won't trigger.

We are done!  Well, except for viewing the results on Google Analytics, which right now there won't be any.  Run a test on your Contact Us form, wait a day or so for Google Analyics to record the event, then check back at Google Analytics, Conversions >> Goals >> Overview to see your goal completion data.  We will go into more advanced Goals and Conversions methods in a future blog post, but for now you working toward a goal.

And be sure to use the FireRoad Digital Contact Us form to see what you get after pressing Send Message!

North Texas Drupal November Meetup

We had a great meetup tonight with lots of interesting discussions on Drupal security, hosting, tools, drush and more.  And a visit from a Drupler from LA!  No meetup in December - we will see everyone in 2015.
 
Here's the notes and links I have from the meetup.
 

Drupageddon related links

Misc

Drush

Hosting

VPN Setup

Cheap bulk emailing

Verify your Drupal Site!

To use most of the major search engines tools and apps with your website (Google Webmaster Tools, Bing Webmaster Tools (which also works with Yahoo! search), Google for Work, others) first you must verify you own the domain.  This can be done in several ways - updating some settings with your domain registrar, uploading a specific file to the root of your website, and setting a specific meta field on the front page of your website are the most common.  The Drupal Site Verification module can help you quickly verify your site using the latter two methods.  Install it first in your preferred manner.

We will look at verifying your site Google Webmaster Tools.  First log in using your Google account.  Next click on the orange ADD A SITE button on the upper right.  You should see this.

Add a site dialog box on Google Webmaster Tools

Enter your sitename and click Continue.  At this point, you might see one of few different screens, depending on what Google thinks is the Recommended option for you to verify your site.  If the HTML file upload method isn't under Recommended method, then click on Alternate methods.

Google Webmaster Tools Recommended method to verify site ownership

To use this method, first click on the link to this HTML verfification file to download it.  Then in another browser tab, we will use the Drupal Site Verification module you installed earlier to upload it to your site.  Navigate to admin/config/search/verifications. Then click on the +Add Verification link.  Select Google from the drop-down list, then click on Next

Drupal Site Verification module configuration screen

You can upload the verification file Google gave you here.  If you chose the META tag option, you can cut and paste it from the Google page to the field here.  Click Save.  Go back to your Google Webmaster Tools tab and click on the red VERIFY button.  It should give you a confirmation screen.

That's it!  You are verified and have the Seal of Approval!  I'll bet you were wondering what the seal cover image had to do with site verification. We're funny, that's what!  Kinda funny anyway.

Drupageddon: Aftermath

By now you've heard about the Drupal core vulnerability commonly referred to as Drupageddon. Let's look at some common questions.

How Bad Was It?

The Drupal community freaked out. Actually, the Internet community freaked out. Was the freakout justified? Unfortunately, the answer is yes. The vulnerability allows anonymous users potentially unfettered access to PHP and the site database and as soon as the Drupal Security Team announced it, the world knew about it. Within seven hours of the announcement, there were already reports of automated attacks.

There was a report that "at least 12 million" sites had been compromised, but the report based this number on bad assumptions, so you should ignore it, much in the way you should ignore numbers in advertisements qualified with "up to" or "results not typical".

What Was It?

Essentially, the vulnerability allowed savvy site visitors to perform arbitrary operations on the site database. There is an excellent post explaining what the issue was if you're interested in the technical details. Once an attacker gained a foothold in the database, then that opened the door to executing arbitrary PHP code and from there, your world was his (or her) oyster.

Has It Been Fixed?

Yes, Drupal 7.32, released in conjunction with the security announcement, fixed it. Update your Drupal core to that version, or at least apply this patch.

Can I Tell If My Site Has or Hasn't Been Compromised?

You can check for some of the known attack patterns to prove that your site has been compromised, but unfortunately, you cannot prove that your site was not compromised. Attacks could have done their dirty work and then left no trace. They could have left a backdoor that didn't follow any of the known attack patterns. As with many things, it's difficult to prove a negative.

What Is The Safest Thing To Do?

The Drupal Security Team released a frightening (and accurate) PSA that recommended rebuilding your server and restoring your site (code, files and database) from a backup made before October 15. Yes, that means what you think it means.

Do I Really Have To Rebuild Everything?

Per the Drupal Security Team, that's the safest thing to do. You don't have to do it, you merely assume more risk if you don't. Fortunately, there are also some potentially mitigating circumstances. If you host your site with one of the big Drupal-centric hosts, then your risk could be reduced if your host rolled out protections along with the security announcement. Both Acquia and Pantheon posted they had done this. Other hosts may have implemented protective measures as well, but the less Drupal-centric your host, the less likely that would be.

You can also check for some of the known attack patterns. While their absence doesn't mean that your site has not been compromised, it could affect your decision with regard to the risk you are taking by not rebuilding.

  • If your source code is under version control (like git), check to see if your codebase has been modified or if there are new files that have been added.
  • Check your uploaded files (which are normally not under source control) to see if there are any PHP files.
  • If you don't have your source under version control, you can still check for any PHP files (not just in the uploaded files, but everywhere). Be aware that there are plenty of legitimate reasons for a PHP file to exist in your codebase, so don't delete indiscriminately.
  • If you have access to your database, either through the command line or a UI, check the menu_router table for suspicious callback values, like file_put_contents, php_eval, or assert.
  • Look for any suspicious users, especially ones with the administrator role.
  • Look for blocks or fields using the PHP Code text format. Inspect them for suspicious content.

It is likely that more patterns will emerge, so keep an eye on your favorite Drupal blogs for updates.

Want more options? Here's a flowchart detailing how you can evaluate your site along with some recommended actions.

Setting up an XML Sitemap in Drupal

Google and the other search engine crawlers do a decent job of finding their way around your website. But, sometimes they could use a little help, as well as to understand the relative importance of your web pages and how often they are updated.

This can be done with an XML Sitemap.  And of course, there is a very popular Drupal contrib module for it - aptly named, XML Sitemap. Installation is easy, but you will need to configure a few things before it starts doing its magic.  We will look at the Drupal 7 version.  And as always, it's good to have a site and database backup before you start.  We are using the currently released version- 7.20.

So, after installing the module in one of the usual ways, a new section in /admin/modules will appear - navigate down toward the bottom :

Enable XML sitemapXML sitemap engines, and XML sitemap node (as shown above).  And Save.

Now we need to prepare our content to be added to our XML sitemap.  Edit the structure of each content type you want added to the sitemap (such as a Basic page - admin/structure/types/manage/page), then click on the XML Sitemap vertical tab near the bottom left.  Then under Inclusion, select Included.  For most pages you can leave the default priority as is, or change it for frequently or infrequently updated pages.  Then click Save.  Do the same for any other Content Types you wish to include.

Drupal configuration screen to include content in an XML Sitemap

When the XML sitemap module was first installed, it will have created a default (but empty) XML sitemap.  Next we need to rebuild it so it will pick up all the new pages we configured the Content Types to include.  Navigate to /admin/config/search/xmlsitemap

Click on the Rebuild Links tab, then when it changes, click on the Rebuild sitemap button (even though it may tell you that you don't need to).  When done, you should see some numbers shown under Links and Pages.  See if this is about right for your site.  You can view more details on pages in your sitemap and priorities under the Settings tab.

The only thing left to do is tell it to submit your new sitemap to the search engines.  Click on Search Engines and check Google and Bing.  Save.  Done!  Wait a day and check on your Google Webmaster Tools account that it is receiving your XML Sitemap.  You don't use Webmaster Tools !?  You should!  It's free and packed full of information from Google on how your site works with its search.  Look for an article here soon on how and why.  You'll notice the module also suggests you verify your site using the Site Verification module.  Good advice!  We'll cover that soon too, although it's fairly easy to use.

North Texas Drupal October Meetup

We had yet another great meeting of the nascent North Texas Drupal users group this past October 20th.

Travis Tidwell asked the question- When Should You *Not* Use Drupal?  While we all love and user Drupal, it is not always the best tool for the job.  Check out his slides below for more info as well as some notes and related links.

Ian Whitcomb took us through building multi-site Drupal, including solutions such as MultisiteDomain AccessOrganic Groups and andPantheon One.

We also had a discssion of Drupal security, including the recent major core exploit.   Here is a way to see if your site was compromised using a drush command.  

Blog article about presentation with Video and Slides

http://travistidwell.com/blog/2014/10/20/when-to-not-use-drupal/­

Static Site Generators

Bootstrapping your site

"API-first" Development

  • Drupal 8 with Symfony is a solid contender
  • Recommend to start simple with Symfony and bring in Drupal if requirements need it.
  • Other API-first platforms

Twitter Issues On Pantheon?

Having trouble getting your Twitter feed working on Pantheon? Does it work locally but not in your Pantheon environments? If your Twitter solution uses PHP's http_build_query function (as does the Rise theme's front page), you're not going crazy. It's them, not you.

The http_build_query function takes an array and returns a URL encoded query string suitable for use by functions that might find query strings handy, like PHP's curl functions. By default, PHP uses the & character to separate query string parameters, but on Pantheon, this value is set to use the HTML entity & instead. If you try to pass a query string with & instead of &, it will fail.

Fortunately, you can override this value using  the third paramter of the call to http_build_query. For example,

http_build_query($my_params, '', '&');

But what if the call is in somebody else's code and you don't want to hack it and cause kitten-death? Well, there is still a potential solution. If you are not relying on the & output from http_build_query elsewhere on your site, you can globally override Pantheon's value with a setting in your site's settings.php file :

ini_set('arg_separator.output', '&');

That's it. No Twitter token regenerating, no hacking, no hair pulling. Just a one line fix.

North Texas Drupal Users Group - Sept meetup

Great meetup of the North Texas Drupal Users Group last night.  The new Addison Treehouse cowork / incubator space was good enough to host us.

First off Kyle Taylor showed off his efforts in integrating the (becoming standard) Bootstrap theme with the core Color module and LESS.  The lead Bootstrap maintainer, Mark Carver, is one of our members and was in house to answer deep Bootstrap questions.

Then Randall Knutson did a presentation on Static Drupal – Taking the Pain out of Drupal Hosting.  It's based on some work he's doing at Phase2, to improve the load times and security of Drupal sites. You can learn more from his blog post and download the Static module on drupal.org.

We're working on the what and where for the October meetup - stay tuned!

HTTPS is now a Google Ranking Factor

You may have noticed that when you use your web browser to do online banking or shopping, and it generally switches to HTTPS / SSL mode (with the little lock graphic) for added security.  Looks like Google wants to make that the standard for everyone soon. According to the Official Google Webmaster Central Blog

"Over the past few months we’ve been running tests taking into account whether sites use secure, encrypted connections as a signal in our search ranking algorithms. We've seen positive results, so we're starting to use HTTPS as a ranking signal."

When HTTPS mode is enabled, all traffic is encrypted between your computer and the web server. Which is generally a good thing, but does add a small performance penalty for the time it takes to encrypt and decrypt all the traffic.  Often this is done on non-ecommerce Drupal websites just for the user login and user editing pages, as most content is really not that security sensitive.  But with this announcement from Google, it's time to start thinking about switching your entire site over to HTTPS.

So, how to do that in Drupal? The complete process is too complex for the scope of this blog post, and you will need to make changes on both the webserver and the Drupal sides.  On the server side, you will need to install an X.509 SSL certificate and make some configuration changes. Many web hosts will greatly help with that process, including our hosting partner, Pantheon. On the Drupal side, the easiest way to enable HTTPS mode is by using the Secure Pages module.

Screenshot of Drupal Secure Pages module configuration

The Secure Pages module will not let you engage secure mode until the server is correctly configured to use SSL.  To best comply with the Google initiative, you should select Make every page secure .. and delete all the pages listed in the Pages box, so that every page runs in HTTPS.

Ironically, that Google blog post (and the entire website) is not running in HTTPS mode.

Pages