At Percona we provide services both Onsite – visiting the customers and Remote – logging in to their systems or communicating via email,phone,instant messaging.
We believe both approaches have their benefits and drawbacks and mixing them right way allows you to get your problems solved most efficient way.
Onsite visits are great as they allow consultant to meet your team in person and great for relationship building. It is great for architecture design and review as you can sit down with the team and use drawing board. It also often allows the best focus both for consultant and for participating team – when consulting visit is arranged it is usually the top priority for some of the staff members which provide consultant with information and assistance he might need.
Onsite visits also often allow to get prompt attention from other team – looking for network engineer to understand network topology or for someone from marketing to get the growth plans – they typically can be brought in for question.
I believe Onsite visits also offer unmatched training experience for the team – one of the team members can work side by side with consultants observing what he does and asking the question about the progress.
The Single Day visits are often extremely valuable as a kick off to do application redesign, for performance review or for starting long term Remote DBA or support relationships. They can be done with little planning as basically for any system there is enough work to spend efficiently understanding the system and working on the pressing problems customer may have. Such visits are often result in a lot of “homework” as application changes or implementing changes on production which when can benefit from remote followups to validate changes and provide further advice.
Multi day onsite visits require a bit more planning to be efficient. Many times in my career I would come to the customer for a week only to find I have created the implementation plan within 1-2 days and it will take a while to implement it and pass it through development and testing stages. Before long onsite visit it is very good to have a call with consultant to understand what exactly needs to be done, how much time and what resources it requires.
Projects which can be efficient as multi day visits are whenever implementation is required (coding unlike giving advice takes a lot of time) – migration projects, any forms of scripting, implementation (writing queries), hands on setting up MySQL, Replication, Monitoring, High Availability with MMM or DRBD are good examples.
Another good example of efficient Multi-Day visit is when there are multiple groups or multiple applications using MySQL – in such case each of the problem areas/groups can gets its own day or few hours which keeps consultant busy.
To make a Multi-Day visit efficient it is important to plan on what will be done – what people will consultant need to work with and which problems do they have. Providing proper access and having staging/test systems available are also often required.
Remote work has a benefit of consultant being available in the short time intervals and being able to multi task.
Working with the customer there are two types of “waits” which are often encountered. First – waiting on the customer. It could be fixing access problems, completing application changes, QA, scheduling downtime to implement changes and millions of other things. Second – waiting on database (or other) operations to complete – backup, restore, ALTER TABLE, load testing – all of these things take time and make consultant to wait for them.
In case you’re working remotely consultant typically has other tasks to do while such long processes are running. If you’re spending time onsite you may have other tasks or you may not while if you’re working remotely there are always things to do.
Another benefit of remote work – it truly allows to engage the team. If you have consultant onsite getting another one with some specific knowledge is complicated while getting another person to login to the system remotely for a quick look is easy and efficient.
Time is another interesting factor – onsite visits usually happen during office hours while changes to production are often avoided. With remote work it is much easier to do work which needs to be done at night hours.
Remote work however also needs to be well organized. The benefit of onsite work is – it is always planned at very specific time. Given date and time consultant will be there so both consultant and customer prepare for the visit. If appointment is just a phone call or online meeting there is larger chance it can be forgotten by ether party. In many cases the remote work not planned to the specific time at all which can cause it to be delayed because of either party delays.
If you need something to be done fast and you’re doing it remotely make sure to plan it to exact time and make sure there is someone to assist consultant if he needs any help such as access issues, some questions about setup or what exactly given queries are doing. Prompt responses can keep the ball rolling much faster.
Another trick to make remote work successful is to be very clear in your instructions especially if you’re not going to be available to assist consultant when he is doing the work. As remote work is often done cross time zones this becomes a very important factor.
In case you have a lot of work done in such “disconnected” mode it is a good idea to have daily/weekly/or be-weekly (depending on project intensity) calls to get team to discuss the progress and generally get team on the same page.
In general surprisingly a lot of things can be done in completely remote mode – we’ve done not only basic optimizations but architecture redesigns, large scale deployment, migrations and high available architectures implementation such a way often surprising customers (in a good way) of how little customers the work took.
The Great Mix As I mentioned before mixing Onsite and Remote work can be indeed the best mix especially for the complex projects. Onsite visit is great to setup relationships, get initial understanding of the system and get initial project planning. After it is done a lot of work can be done remotely quite efficient while there may be the benefit in onsite status-checks and team interactions as well.