VCAP6.5-DCV Design Exam Experience

This year, I got the opportunity to go to VMware Empower 2019 in Lisbon. The ticket includes a free exam voucher so I used it to take my VCAP6.5-DCV Design exam. I’m glad to say that I passed! The exam was a lot harder than I expected though.

Having just done the VCP exams, I did not expect the VCAP to be that much harder. In this post, I will try to share some tips that I found useful during preparation and how I experienced the exam

VCAP

Let’s start by clarifying what a VCAP actually is. VCAP stands for VMware Certified Advanced Professional. VMware qualifies the “minimally qualified candidate” as follows in the exam blueprint

A minimally qualified candidate (MQC) achieving the VMware Certified Advanced Professional 6.5 in Data Center Virtualization Design is capable of developing a conceptual design given a set of customer requirements, determining the functional requirements needed to create a logical design, and architecting a physical design using these elements.

This quote, together with the exam title, give a clear picture of where the focus for this exam is. You are also expected to know all the topics that were handled in the VCP exam and in much greater detail. After all, this is an advanced certification exam.

Preparation

During my preparation, I read a lot of whitepapers and articles. Below you can find a list of the ones I used. Don’t limit yourself to just this list, there’s a ton of content out there created by fellow bloggers and VMware itself. Also, make sure to read other posts like this. Most people also link to content that helped them during their preparation.

Official resources

Community resources

  • Graham Barker’s VCAP prep guide is a must read for everyone in my opinion. It’s written for the VCAP6 exam but the design parts still apply to 6.5
  • David Stamen’s tips helped me a lot during my exam, definitely recommend reading them!
  • the vMusketeers created a VCAP6-DCV Design quiz that you should try for sure. I found that test to be harder than the exam itself. It really tests your knowledge on the core design concepts.
  • Daniel Paluszek created an extensive blog post with a lot of material in it. I used it a lot.
  • Jeffrey Kusters’ blog on designs, requirements, etc. really helped me understand these concepts

Exam experience

For me, this was the hardest exam that I’ve done so far. Questions were multiple choice style and also drag and drop style. Obviously, I can’t go talk about the questions but the things I can tell are that you should really be familiar with these concepts

  • RCAR (Requirements, Constraints, Assumptions, and Risks)
  • AMPRS (Availability, Manageability, Performance, Recoverability, Security)
  • RTO (Recovery Time Objective)
  • RPO (Recovery Point Objective)
  • All vSphere features. design and architecture of all components should be known
  • all vSAN features, how to design it and how the architecture works
  • Replication, SRM, …
  • Conceptual, logical and physical design

Now that I passed the exam, my focus will shift to preparing my upcoming VMUG session and afterwards the VCAP6.5 deploy exam. Exciting times ahead!

I hope that this post gives you a good idea of just how much preparation is required to pass the exam. If you have questions or additions, hit me up!


How to reset iLO password from ESXi

Everyone’s been here before, you know the iLO password should be set to something, only to find out that on just this one server the password is different.

There are several ways to reset the iLO password; you could reboot the server and reset it that way. But who likes planning downtime on a server for a password reset? Luckily you find that the server is an ESXi host that was installed using the HPE customized image. This is good news as you can now use the HPONCFG tool to reset the password!

Checking the configuration

Before you start resetting the administrator password, you can always check the current configuration. This will show you the entire configuration done on the iLO, including any additional users that were created.

To start using the HPONCFG tool, first enable SSH on the ESXi host in question and log on. Once logged on, go to /opt/tools . This is where you will find the HPONCFG tool.

To export the current config, run this command to write everything to a file

You can then look at the file using your editor of choice. In the file you can see the IP address assigned, DNS name of the iLO and, most importantly, the users created. You can see here that there’s 3 users

Resetting the password

Now that we know which users exist on the iLO, we can start doing the actual password reset on the user we want. To do this, we will create an XML file on the ESXi host that contains the new password for the user.

Create the new file on the host, using your editor of choice. I like to use vi, so that’s what you will see below.

Now you can press the i key to insert, copy the code below and right click in your ssh window to paste. You can replace the [email protected]! with the password you want to set on the user

Afterwards, hit the escape button and type :wq to save the file. Now we can start the actual reset. Run the command below to perform the reset

If the reset is successful, you should see the output like this

Cleaning up

Now that the password has been reset, login just to make sure everything is working as expected. As a security precaution, make sure you delete the pwreset.xml file, once you have verified you can log in! We don’t want to store passwords in clean text anywhere 😉

The above command will delete the file, if you’re still in the /opt/tools folder.


VMUGBE

Next week, the 14th of June, it’s time for one of my favorite times of the year, the yearly VMUGBE is upon us! It’s an amazing day filled with great sessions, good conversations, all set in a lovely venue.

The Belgian VMUG was the first VMUG I ever attended, 4 years ago already. It’s where I met a lot of people I know in the community and, on a personal level, where I learned to just go up to random people and start talking to them. Something I’m still not very good at but there’s plenty of VMUGs to come!

Why you should attend

VMUGBE+ is a free event that gets top-notch speakers every year with topics ranging from more traditional topics to newer topics like CNA, VMware cloud on AWS and so on.

Coming to the VMUG is also a great opportunity to meet new people that are a part of the same #vCommunity we are all part of. Undoubtedly, everyone has found a solution to their problem on the blog of someone from the community. This is your chance to talk to them and perhaps even thank them!

Since 2013 VMUGBE has become VMUGBE+, which means that the organization will donate an amount of the earnings to a charity in Belgium. This ties in nicely to the whole idea of giving back to the community. You can read more about it here.

Sessions I’m looking forward to

As I said before, VMUGBE has a lot of great sessions but there are a few that I’m really looking forward to attending!

Opening keynote – Joe Baguley – 4 years ago the keynote was also given by Joe Baguley, I still remember it very well. He is the CTO for EMEA and will be talking about how architectures are changing and some new areas of R&D within VMware

VDI by day, compute by night – Johan Van Amersfoort – I have missed this session at the NLVMUG and Empower in Lisbon, there’s no way I’m missing it now!

Kubernetes for the vSphere admin – Eric De Witte – This is a topic that I don’t know a lot about. Looking forward to getting a better understanding of Kubernetes in this session. I also heard it involves rockets!

vExpert Community Session – Stijn Depril – One of the sessions I’m looking forward to the most! Stijn is one of Belgium’s vExpert Pro and a big advocate for the community. The goal of this session is to educate people on what a vExpert is and to get people excited about becoming a vExpert!

Presenting

This VMUG will be special for me, as I will be presenting for the first time! VVOLs is something that’s been talked about since 2011 but to many people it’s still shrouded in mystery. The goal of my session is to dispel some of this mystery and get people excited about what VVOLs can do for your environment.

As you can see, there’s a lot to look forward to. So what are you waiting for? It’s not too late to register! https://www.vmug.be/#register

Image result for arnold do it now

I look forward to seeing you all there!


Rescanning ESXi storage in a parallel way

Lately, I’ve been doing a lot of work provisioning and decommissioning LUNs on some Fibre Channel arrays. As you all know, this process can be somewhat tedious when provisioning a new LUN on vSphere or removing said LUN.

In the past, I would always use this PowerCLI command

This would first get all the ESXi hosts in the cluster “Cluster1” and then launch the refresh cmdlet on each host. The problem with this is that it can take a really long time for the command to complete because the refresh is executed host by host. Before starting the next refresh, the command waits for the previous one to finish. While this is fine for smaller environments, the time really does add up when you have a decent sized cluster.

Using PowerShell jobs

In order to reduce the time this process takes, I decided this would be a great use case for PowerShell jobs. A job is essentially another PowerShell instance that runs in the background. This means that your main script or PowerShell window doesn’t wait for it to finish before going to the next step. Using jobs will allow us to start the rescan on each host as a job and then start the next job. Meaning the rescans will happen in parallel.

Code

The script is provided as is and can be found on GitHub


ERROR: The host returns esxupdate error code:15

Today I was preparing some updates with VUM on Lenovo servers running vSphere 6.0. Things did not go as expected. While staging the patches I was greeted with this error.

ERROR: The host returns esxupdate error code:15. The package manager transactions is not successful. Check the update Manager log files and esxupdate.log files for more details.

Looking into the esxupdate.log file I could find the following entries:

This seems to be a known issue for Lenovo servers. It turns out they cram a whole lot of drivers in there, more than other vendors. The only known workaround is to remove VIBs from the host that are not needed. This is also described in https://kb.vmware.com/s/article/2144200.

To find out what VIBs are installed, log onto the affected host using SSH and run this command:

Once you have identified a driver that you want to remove, put the host in maintenance mode and this command:

In my case I wasn’t using the qlogic nx2 driver so I removed it. Now you should be able to update the host using VUM,


Event ID 1000 rundll32.exe_aepdu.dll

While doing my morning check of our monitoring system I came across a strange issue…Two of our servers had been unavailable during the night, this had also happened the night before. Time to dig a bit deeper!

Both servers are domain controllers running Server 2012 R2, no updates were installed in the last few weeks and nothing had been changed either. Pulling up the event log of both servers, I found the exact same event logged on both machines at the time they were unavailable to our monitoring system.

After I quick search, I found out that aeinv.dll is part of the Microsoft Customer Experience Improvement Program (CEIP). If you’re signed up to the CEIP, which you are by default when you install Server 2012 R2, three scheduled tasks will run each night to upload your anonymized date to Microsoft.

  • AitAgent
  • Microsoft compatibility Appraiser
  • ProgramDataUpdater

We had the unavailability after the first and third task ran.

Luckily you can easily opt out of the CEIP by opening Server Manager, going to Local Server and clicking on the link behind Customer Experience Improvement Program.

There you will see that the radio button, Yes, I want to participate in the CEIP is selected. Click on No, I don’t want to participate and click OK.

You’re now opted out and the scheduled tasks will no longer run.

UPDATE 11/07: Turns out that just opting out of the CEIP does not actually prevent the scheduled tasks from running. In order to fully disable them, you need to disable them through the task scheduler or, even better, disable them through PowerShell


Quick post: Bug in Veeam Quick Migration

Last week I was moving some VMs for a client using Veeam Quick Migration. During the migration, I happened to stumble upon some strange behavior.

The source VMs make up an Oracle RAC cluster, using shared VMDKs. One of the requirements for setting up shared VMDKs, using the multi-writer option, is that they are thick eager zeroed disks. When the VMs got to the other side, they wouldn’t boot. I figured something went wrong during the migration so I tried it again. Once the second run completed, the VMs still wouldn’t boot so I started digging around some more.

The error message I was getting was rather vague; “Incompatible device backing specified for device ‘0’.”. After verifying the config of both nodes I eventually decided to look at the disk type on the destination side. That’s when I noticed the disk type was thick provisioned lazy zeroed. Ahah, that’s why they didn’t want to boot! After manually inflating the disks, they were up and running again. I’m starting to suspect that this is a bug.

Running some tests

I started building some more test VMs just to prove that this was, in fact, a bug. One of the options you can set during the Quick Migration wizard, is the disk type. You can explicitly select each of the types, or you can have Veeam use the same format as on the source side. Explicitly selecting thick provisioned eager zeroed or using the same as source also produced a VM with lazy zeroed disks. Time to submit a ticket!

As usual, Veeam support was very helpful and investigated the issue. A couple days later they came back to me and confirmed this was indeed a bug that will be fixed in an upcoming version.

Workaround

This bug is a minor inconvenience since there is an easy workaround. You can login to an ESXi server using SSH and convert the VMDK using the command

More info on how to convert a VMDK can be found in this VMware KB.


Publicly share Office 365 room calendar

A customer asked me if it was possible to have a room mailbox automatically accept meeting requests from external parties. They would also like to publish the calendar of that specific room publicly.

Accept meetings from external parties

Let’s start with the first question. By default, resource mailboxes only accept requests from internal senders. As you might guess, you can’t change this behavior through the GUI, Powershell to the rescue!

Since I didn’t know the cmdlet that would let me change this behavior, the first thing I did was look for all “Calendar cmdlets”. After connecting to the Office 365 PowerShell, I ran this command

Seems like there are a few cmdlets concerning calendars, good info for the second question! The Get-CalendarProcessing cmdlet looks promising, let’s try it out!

As you can see on the highlighted line, this is exactly the property we were looking for. Let’s change it so we get the desired behavior. In the get-command output, I saw a cmdlet Set-CalendarProcessing, this seems like the right one.

This change will only affect new meeting requests, requests that have already been refused won’t be automatically accepted.

Publish calendar publicly

In the cmdlets we got earlier, there wasn’t really one that stood out as a “possible match” so let’s look at the attributes of the calendar itself. In essence, the calendar is just a folder inside of a mailbox object. Let’s query that folder directly.

That’s everything we need and more! As you can see, we can set the PublishEnabled attribute to true but we can do so much more. You can choose the detail level and even set how far back and forth the published calendar needs to go.

Let’s publish the calendar and run the Get-MailboxCalendarFolder cmdlet again to get the URL.

All done! Now you can browse to the URL and verify everything is being displayed as you’d expect.


Cannot get extent connection. Failed to restore file from local backup

When I got into the office this morning, I noticed that on particular copy job hadn’t done its` job over the weekend. This particular job copies the daily restore points to a separate scale-out repository and enforces the GFS scheme that’s been set.

The job report displayed

Not that much to go on if you ask me. First, I checked to see if all extents in the repository still had enough room, this was the case. While I was doing that, I verified that all my proxies were still up and running. Before heading to my good friend Google, I decided to remove the copy job restore points from the configuration.

After this, I did a rescan of the repository and retried the job. It ran without a hitch, a nice and easy fix 🙂 I hope this won’t become a common thing, time will tell.


Looking forward to the coming year

It’s been fairly quiet on the blog front lately, with this post I’m trying to pick it back up again 🙂 I decided to put the blog in a fresh new theme, I really like how it turned out! It looks a lot cleaner now.

2016 in review

When I started this blog last year, I made a goals page that lists everything I wanted to get done that year. A quick review:

  • Earn the VCP6-DCV certificate –> FAIL. I only managed to get part 1/2 done, I haven’t gotten to part 2 yet, but it is one of the top priorities this year.
  • Become CCNA  –> FAIL. I will be taking this one off the list. When I have some more time, I might pick it up. But for the moment, I feel my knowledge is good enough to get by.
  • Learn PowerShell –> PASS. Last year I started performing routine tasks using PowerShell. Eventually, I was able to automate some of the things that I had to do frequently. Over the past year, I’ve gotten a pretty good feel for the language and I’m constantly discovering new things! I will be continuing with this until I feel that I have mastered it.
  • Publish code to GitHub –> PASS. I published 2 scripts that I created. I also made my first ever pull request and added some tests to the awesome Vester project (If you don’t know it, check it out here!). When I get some more free time, I will be looking for some more projects to contribute to.
  • Attend VMUG(s) –> PASS. I managed to go to both the Belgian as the Dutch VMUGs and I’m hoping to do it this year as well.

2016 was also a big year on a personal level. In august, I changed jobs and started working for Realdolmen as a system engineer. This was one of the best decisions I have made recently, working here gives me the chance to interact with some of the smartest people I know. I get to work with complex and interesting environments and I’m learning new things every day!

But the most important thing I did, was asking my girlfriend to marry me. She said yes and we’re getting married this coming May, I’m very much looking forward to it!

Looking forward to 2017

Obviously the biggest thing for me this year is my wedding. Shortly after that, 29 colleagues and I will be climbing the legendary Mont Ventoux by bike. I’m riding a lot again and found the joy of cycling again.

On a professional level, I will also be setting a few goals for the coming year.

  • Earn the VCP6-DCV certificate –> This is the first thing I want to get done education wise this year. It’s possible I won’t have time to do this until the summer, though.
  • Continue learning PowerShell –> There’s a ton I don’t know yet, and a lot that I can do better. I’ve started to put most of my code in functions and I will be looking into building some modules where I can.
  • Upgrade my MCSA to 2016 –> With the release of Server 2016, it’s time to upgrade my MCSA. I don’t want to let it expire, which would mean I would have to take the first 3 exams again.
  • Keep the blog more active this year –> Changing jobs in the summer, starting cycling again and preparing for our wedding has eaten up most of my free time since august. I’m hoping to find some more time to keep this blog going with some new content!
  • Wildcard –> I’m keeping this one open for something else to do the coming year. I’m not entirely sure what it is yet, it will all depend on the amount of free time I have and how the other goals have come this year.

A short list this year, but with a lot going on in my personal life, this feels reasonable.