Virtual SAN is breaking down silos for United Utilities

Last week I had the pleasure to catch up with Rob Alford from United Utilities. For those who don’t know, United Utilities is the largest listed water company in the UK, based in the north west of England.

I met Rob at VMworld last year so I knew they were working with Virtual SAN and had actually bought Virtual SAN on day 1 of GA, one of the first VSAN customer in Europe for sure and probably worldwide. As I mentioned they are a utilities company, and as you can imagine they run really mission critical workloads, service disruption is not something they can afford. And yes, some of these workloads sit on top of Virtual SAN. I found interesting to hear the approach United Utilities took. Simply said: Virtual SAN first approach. A policy very similar to the “virtualization first” approach many of you had years back.

I wondered what got Rob and his team interested in Virtual SAN. Rob mentioned they have a relatively large environment (2500+ VMs) and a big believer of “converged infrastructure”. They’ve been running on converged infrastructure for a long time and they feel the hyper-converged is that next step that will offer them more flexibility against lower cost. (More explicitly, implementation of VSAN resulted in 50-60% cost reduction for United Utilities.) The big challenge with “traditional converged solutions” for them was most definitely the huge up front cost and scale. Virtual SAN offers both scale up and scale out per host, making it far more flexible than any other solution out there. Rob mentioned they had looked at other hyper-converged solutions but weren’t convinced by running a Virtual Storage Appliance on top of vSphere, they wanted to keep things simple and as close to the hypervisor as possible. When they heard first about Virtual SAN they were sold, they loved the concept of having a storage solution tightly integrated with their virtualization platform.

United Utilities was already a big VMware customer and one of the bigger challenges they’ve faced (and are still facing to a certain extend) was much more organizational than it is technical. They have a traditional IT team and are very much siloed. They have a big VMware team, with a lot of in-depth knowledge and Virtual SAN helped them to get around these silos. It did mean the team had to think about things in a different way as Virtual SAN introduces some new concepts. Nothing they couldn’t overcome but they did notice that things like for instance disk controllers are all of a sudden really important.

Key take away: Spend the time upfront researching hardware options, educate yourself / team as importance of components may change.

I asked Rob how Virtual SAN was holding up, if there were was anything that surprised him. He told me they have a billing system which holds between 6 to 7 million customers. Their billing cycle is a slow process and running on a physical machine with a high-end enterprise storage system takes about 22 hours to complete end-to-end. When they started testing the same billing process on a virtual environment, while still leveraging the same storage platform, they were capable of reducing the 22 hours down to 14 – 16 hours. A big win for Rob and the team. Of course the team wanted to know what would happen when this same workload would run on top of Virtual SAN, they were hoping to get at least the same as when using the high-end enterprise storage solution and I guess it is fair to say that they were shocked when they saw that the exact same run completed in under 3 hours. Yes, down from 22 hours to under 3 hours is a huge win indeed.

When I asked Rob what their environment looked like he said they have over 2500 VMs still on legacy converged infrastructure and over 650 VMs on Virtual SAN. Plan is to migrate those 2500 VMs over soon, in the upcoming 12 months. As they are a very traditional company they also still have many physical machines which they are also hoping to virtualize soon. A big undertaking and of course “Virtual SAN first” policy applies here.

I asked Rob what their Virtual SAN hosts looks like. Rob said they have clusters up to the size of 8. They’ve always tried to keep failure domains small and applied the same logic to Virtual SAN, strictly speaking from a VSAN approach this isn’t needed but I can understand why they do it. Each host in those clusters has 5 disk groups, with each disk group having a 200GB SSD and three 1.2TB SAS disks. That is 15 disks per host with a total of 18TB worth of storage capacity and 1TB worth of flash. Currently United Utilities uses Dell R730XD for their VSAN clusters, but as Rob stressed the main reason for going hyper-converged and software based is to have the ability to change hardware when they want. All logic should reside in software and the software solution should not limit them to a couple of options but instead provide them flexibility to chose whatever they want.

Key take away: Hardware should be easily replaceable. All logic should reside in software to avoid lock in as much as possible.

We also discussed how United Utilities handled things like providing extra availability for those services needing it. Rob mentioned that they have services which from an application standpoint are clustered, but also have apps which are not and for those apps United Utilities uses vSphere HA and Veeam Replication. Not just for DR, but also for workload mobility. Nice solution if you ask me, and once again not tied to any hardware platform.

Before we wrapped up Rob mentioned one other thing he was impressed about, when they started working with Virtual SAN they had issues with the driver/firmware of their disk controller. What surprised Rob was how engaged the engineering team was, direct contact with engineers to see how the problem can be resolved is not something you experience too often.

I want to thank Rob for taking the time to provide some more insights in their journey and infrastructure, and I think it is safe to say that Virtual SAN is breaking down silos for United Utilities!

My first week with Pebble Time

When Pebble Time was announced I backed the project on kickstarter based on Joe Baguley and Alan Renouf’s stories. Last week I received it and I’ve played with it a lot I must say and have been impressed from the moment it arrived. Simple things like the box it comes in (hardly any packaging) to the ease of use and how it feels / wears. Lets go over some of the things that stand out for me personally and why I like the watch.

As soon as it arrived I downloaded the Pebble Time app for iOS and added some apps and watch faces. I’ve ended up going with a more classic watch face called Thin for a couple of days, and then switched to Trek V3 which has been on there for 4 days now. I also like “Stock Weather” which shows time/battery/weather and stock details by default. I am looking for a good dual timezone face still which allows me to show time of the current location and the time at “home” so that I always know if I can phone home or not. Changing watch face is really easy, so when I travel I could swap from Trek V3 to a dual timezone face. I may just end up creating something myself as that doesn’t appear to be rocket science either.

One thing I wasn’t completely sold on was the “look” of the watch, and I wasn’t too convinced of how it would feel / wear either. I normally don’t wear watches as they annoy me during work etc. The pebble feels light (42.5 grams) and looks good in my opinion but I do think it is one of those designs which you either like or you don’t. The strap is rubber based and comfortable. I wear the Pebble 24hrs a day. During 15k hikes, during desk work, during crossfit etc and not once did I take it off, which I used to do with other watches.

** Update – June 19th** Literally two hours after I published this I scratched the bezel of my Pebble. I wasn’t doing anything special and didn’t hit concrete or a rock wall or anything like that. Seems like it scratches easily and if you do a search on google for Pebble Time Scratces you will see more users reporting it. Also seem like the screen scratches easily, I just ordered screen/bezel protectors from gadgetwraps. Hope Pebble will fix this over time. ** Update – June 19th**

First App I wanted to install was runkeeper, but for some strange reason it wouldn’t install… It took me a while to figure out that Runkeeper is integrated with the Pebble Time to the extend where you don’t need an app. As soon as you open the Runkeeper app on your phone the screen on your watch changes and it shows: distance, time and average pace. My feedback to Runkeeper was to also show the current pace, and I would also love to see the stats per kilometer after you have finished a kilometer. Especially when running this is useful as it allows you to keep pace. You can pause and start your run using the Pebble, you can’t stop it… you will need to grab your phone for that. Other than that it appears to work great, and the fact you don’t need to wear a headphone to hear your stats or grab your phone to check your stats is worth it by itself. There was a difference in distance and average pace as well compared to the Garmin, but it was negligible if you ask me. Some more testing showed though that Runkeeper/Pebble seems less accurate than a dedicated GPS tracking device like Garmin.  (Not the only person who has noticed it.)What I like though about the Pebble is that if for whatever reason someone needs to contact you they actually can and you can decide if you stop your workout or finish it before you respond to an email / text / phonecall. Notifications I guess is what I like!

Talking about notifications, this is of course is a big thing on the Pebble, email / twitter / calendar / whatsapp / sms and much more. I tried to keep it to those 5 though as I didn’t want to find myself constantly dismissing notifications. I use tweetbot and it comes in handy I can specify that I only want notifications for direct messages and mentions from people I follow. This restricts the number of notifications I get. This was one of the big reasons I bought a Pebble by the way. I used to be on my phone checking notifications all the time, and then immidiately responding to email/twitter/facebook etc. Considering the number of messages I receive it resulted on basically holding your phone in your hands for hours a day, driving my family nuts. I can’t respond to anything on my Pebble really, so that prevents me from responding to things which can wait. It is nice that after ~3 minutes notifications are automatically dismissed, would be even better if this was configurable though and I submitted that as a feature request to the Pebble team.

First night wearing it I realized that it would be useful to setup “do not disturb” on my Pebble like I have on my iPhone as it woke me up twice. (I have it setup to vibrate on a notification.) Luckily “do not disturb” is easy to setup. It is described in this article, but in short: go to settings, notifications, do not disturb. Works great and solved my problem. Talking about sleeping, there’s also apps for that of course. I figured I would try a “sleep monitoring” app. I installed Morpheuz and configured the smart alarm which wakes you at an optimal point during your sleep cycle (not during the REM / deep sleep). So far it woke me up at various times between 07:00 and 07:25, I am not the type who has problems getting out of bed to be honest but it seems like it is even easier using Morpheuz. Will continue to use it for a while to see if it really makes a difference or not.

Some other apps which are worth mentioning are Evernote, Transport and TripAdvisor. Evernote is primarily for me to quickly dig up that note and read it. Don’t expect to use it a lot, but can be useful every now and then. TripAdvisor can be useful when looking for a place to eat in the area, couple of clicks and you know what can be found relatively closeby. Transport is probably the most useful of the three, as it will allow you to request an Uber ride from your Pebble. There are plenty of apps you can install / try on the Pebble appstore and I suspect that considering the massive number of Pebble watches sold this will increase steadily.

Last but not least battery life. When it comes to battery life the Pebble is pretty impressive. I’ve played with it extensively the last week and I reach about 4-5 days, Pebble promises up to 7 days with normal usage, but considering the number of notifications I get and how extensive I used it I think 4-5 days is pretty good. It charges pretty fast as well, and it comes with a magnetic usb charging cable which is easy to use and carry with you if needed when traveling.

So I guess the question remains if I would recommend it? Yes, I most definitely would. I like the Pebble watch so far, and I think it would be worth it even if you use it for just notifications!

I will see you at VMworld 2015!

Just got the word that I am going to be part of three sessions at VMworld this year, and still waiting on news about the quick talks I submitted. So if you are going to VMworld 2015 make sure to note down these session IDs:

  • SDDC5027 – VCDX Unwrapped – Everything You Wanted to Know About VCDX (US only)
    The VMware Certified Design Expert (VCDX) program is growing every year. More and more people are interested in what it takes to become a VCDX. This moderated talk-show style panel session made up of a VCDX from each of the four tracks, (DCV, NV, DTM, CMA) will help potential candidates understand the value of getting their VCDX. It will also be a no-holds barred open discussion on what it takes to achieve this premier VMware certification. Hear from these experts on their journey and the incredible value that comes with becoming a VCDX yourself. This session will also feature live Q&A from the audience so come prepared with your own questions! This is the place to find out everything you ever wanted to know about becoming a VCDX, from live VCDX holders in a lively interactive session. Featuring: Chris Colotti, Simon Long, Jason Nash and Matt Vandenbeld.
  • SDDC4593 – Ask the Expert vBloggers (US only)
    Back on stage with Rick Scherer, Chad Sakac, Scott Lowe and for me the first time with Chris Wahl! 8th year at VMworld, awesome panel of the industries top bloggers. In this session there are no powerpoints, no sales pitches and no rules! Experts in the industry are here to answer the audiences questions while having some fun in the process. Bring your topic, anything from Software-Defined Data Center, End-User Computing to Hybrid Cloud… Storage, Networking, Security. No questions are off limits.
  • INF4535 – 5 Functions of Software Defined Availability (US and EMEA)
    Together with my friend Frank Denneman… Long time since I’ve been up on stage with Frank, and this VMworld we will be looking at Software Defined Availability. We will discuss 5 functions of Software Defined Availability, which are part of vSphere 6.0. For each of these functions certain scenarios will be discussed to explain how vSphere can help improving availability of your workloads. This ranges from “how Site Recovery Manager and Storage DRS are loosely coupled but tightly integrated” with vSphere 6.0 to “how vSphere HA responds in the case of a certain failure”. Be prepared to get in to the trenches of workload availability

No one ever got fired for buying IBM/HP/DELL/EMC etc

Last week on twitter there was a discussion about hyper-converged solutions and how these were not what someone who works in an enterprise environment would buy for their tier 1 workloads. I asked the question: well what about buying Pure Storage, Tintri, Nimble or Solid Fire systems? All non-hyper converged solutions, but relatively new. Answer was straight forward: not buying those either, big risk. Then the classic comment came:

No one ever got fired for buying IBM (Dell, HP, NetApp, EMC… pick one)

Brilliant marketing slogan by the way (IBM) which has stuck around since the 70s and is now being used by many others. I wondered though… Did anyone ever get fired for buying Pure Storage? Or for buying Tintri? What about Nutanix? Or VMware Virtual SAN? Hold on, maybe someone got fired for buying Nimble, yeah probably Nimble then. No of course not, even after a dozen google searches nothing shows up. Why you may ask yourself, well because typically people don’t get fired for buying a certain solution. People get fired for being incompetent / lazy / stupid. In the case of infrastructure and workloads that translates in to managing and placing workloads incorrectly or misconfiguring infrastructure. Fatal mistakes which result in dataloss or long periods of downtime, that is what gets you fired.

Sure, buying from a startup may impose some risks. But I would hope that everyone reading this weighs those risks against the benefits, that is what you do as an architect in my opinion. You assess risks and you determine how to mitigate those within your budget. (Yes of course taking requirements and constraints in to account as well.)

Now when it comes to these newer storage solutions, and “new” is relative in this case as some have been around for over 5 years, I would argue that the risk is in most cases negligible. Will those newer storage systems be free of bugs? No, but neither will your legacy storage system be. Some of those systems have been around for over a decade and are now used in scenarios they were never designed for, which means that new problems may be exposed. I am not saying that legacy storage systems will break under your workload, but are you taking that risk in to account? Probably not, why not? Because hardly anyone talks about that risk.

If you (still) don’t feel comfortable with that “new” storage system (yet) but they do appear to give you that edge or bigger bang for the buck simply ask the sales rep a couple of questions which will help building trust:

  • How many systems are sold world wide similar to what you are looking to buy and for similar platforms
    • If they sold thousands, but none of them is using vSphere for instance then what are the chances of you hitting that driver problem firsts? If they sold thousand it will be useful to know…
  • How many customers for that particular model
    • Wouldn’t be the first time a vendors sells thousands of boxes to a single customer for a very specific use case and it works great for them, just not in your particular use case.
    • But if they have many customers, maybe ask…
  • If you can talk to a couple of customers
    • Best thing you can ask for in my opinion, reference call or visit. This is when you find out if what is promised actually is reality.

I do believe that the majority of infrastructure related startups are great companies with great technology. Personally I see a bigger threat in terms of sustainability, rather than technology. Not every startup is going to be around 10 years from now. But if you look at all the different storage (or infra) startups which are out there today, and then look at how they are doing in the market it shouldn’t be too difficult to figure out who is in it for the long run. Whether you buy from a well-established vendor or from a relatively new storage company, it is all about your workload. What are the requirements and how can those requirements be satisfied by that platform. Assess the risks and weigh them against the benefit and make a decision based on that. Don’t make decisions based on a marketing slogan that has been around since the 70s. The world looks different now, technology is moving faster than ever before, being stuck in the 70s is not going to help you or your company compete in this day and age.

VMware Online Technology Forum recording available

For those who weren’t able to attend the VMware Online Technology Forum live, the recordings are available for replay now including the Q&A sessions. If you are interested in what’s new for vSphere, VSAN, NSX, vRealize Operations, View and much much more make sure to tune in. Great way to get an overview and a better understanding of what vSphere has to offer. Click the image below to go to the OTF website, register (if you haven’t yet) and get access to a lot of free cool content!