How are IT leaders using automation technologies? What are the keys to success? What’s next? Our comprehensive guide shares advice from CIOs and IT automation experts.
Why is IT automation a growing trend?
How can I get started with IT automation?
What best practices should we use for IT automation?
How can I make the case for IT automation within my organization?
What are some common mistakes with IT automation?
What about IT automation and security?
What’s next in IT automation?
What is Robotic Process Automation (RPA)?
Where can I learn more about IT automation?
In enterprise IT shops, automation has become more than a trend: It has turned into a necessity. That’s due to the current mix of technologies in use, the speed at which IT must run, and talent concerns.
On the technology side, cloud, containers, and microservices lead the list of factors driving increased use of automation. Containers and microservices help tame the complexity of hybrid cloud environments. As you scale up your use of containers and microservices, automation soon becomes a core need.
“Automation gives the administrator tools to effectively deploy scalable workloads, without a commensurate increase in staff,”says Ned Bellavance, director of cloud solutions at Anexinet.
That’s where automation tools like Ansible and orchestration platforms like Kubernetes come to the rescue: They help IT leaders reap the potential of cloud, containers, and microservices in a scalable manner without constantly throwing new bodies at evolving operational needs. (Read our deep dive: Kubernetes: Everything you need to know. )
[ Get the free eBooks: Managing IT with Automation and O’Reilly: Kubernetes Operators: Automating the Container Orchestration Platform. ]
“Developers are becoming acclimated to building and shipping containers. Containers allow for complete packaging of services to eliminate the toil of maintaining unique developer environments," says Chris Short, Red Hat OpenShift principal technical marketing manager. "At the same time, operations teams are gaining the skills and tooling to manage Kubernetes clusters at scale. As those muscles get stronger, teams realize that multiple, smaller clusters meet their demands better.”
“We are quickly moving towards a world where administration of 'legacy' infrastructure can occur in Kubernetes clusters like we see with OpenShift virtualization," Short says. "Kubernetes is looking to become a real powerhouse in enterprise IT for years to come.”
[ Kubernetes terminology, demystified: Get our Kubernetes glossary cheat sheet for IT and business leaders and our shareable primer: How to explain orchestration in plain English. ]
Why is IT automation a growing trend?
The speed at which organizations now work, and the concurrent rise of the DevOps way of working, also fuel the need for automation. DevOps requires paying down technical debt, such as time-sucking manual patches, so people can focus on more important problems.
Since DevOps prizes experimentation, speed, and iteration, the more that teams can automate work, the better. "Engineering practices such as Continuous Integration, Continuous Delivery and Everything-as-Code employ automation at the foundation and [DevOps] teams should seek opportunities to automate as much of their workflow as possible," notes Tim Beattie, Global Head of Product for Red Hat Open Innovation Labs.
Finally, there’s the IT talent factor. As we’ve noted, “If your engineers and other talent spend the bulk of their time on painful, manual tasks, are you getting their best work? Probably not, and that’s a killer when you’re constantly challenged to do more with less."
"In the enterprise IT space, one of automation’s biggest use cases is ensuring businesses have the right people performing the right tasks," says Mark Kirstein, VP, products at BitTitan.
IT leaders need talent working on strategic priorities today, not drudge work that can be automated. Also, IT people whose companies keep them on drudge work are more likely to leave - for a more innovative environment where they can grow their careers. Automation, in this regard, can help with retention.
How can I get started with IT automation?
Perhaps you have already started down the road to automation and are trying to increase its use. Or, perhaps you are just beginning to transition from older, manual ways of working to more automated ones. One of the hardest tasks is knowing what task makes a good automation candidate and what doesn’t.
Before you start automating, or automating more, make sure you are not automating a business or IT process that no longer makes sense, notes Red Hat chief technologist E. G. Nadhan. IT leaders working with automation must keep an eye on processes that can be sunsetted instead of automated.
“Automation is an effective trigger to revisit existing processes and determine their applicability in today’s market,” he writes. “While it is true that repeatable processes warrant automation, it is important to make sure they’re being executed the right way. Automating the wrong processes proliferates chaos.”
After examining your processes, identify types of low-skill work that vacuum up your IT team’s time.
As ServiceNow CIO Chris Bedi recently wrote, IT teams continue to spend too much of their time managing the day-to-day tasks involved in keeping the business up and running.
Bedi sought to free up more of his team’s time for innovation – after finding his team was
spending nearly three-quarters of its time on day-to-day operations. So he decided to go all-in on automation. He advises an incremental approach – and patience.
“To be a 'no service' organization will take some time, effort, and a plan,” Bedi writes. “Service automation can’t just be about quick wins and incremental improvement. It’s about creating competitive advantage over the long run: It’s a marathon, not a sprint. You won’t jump from manual to machines completely managing every service in one step.”
“Start with end-to-end processes with a lot of structured tasks and where automation will alleviate the most workload for the IT team. Provisioning VMs, patching machines, and installing are good candidates. Automate these entire processes – the provisioning, the management, the reporting, the scaling up and down. Learn from them, then tackle more.”
Bedi’s team boosted its operations resources spent on innovation from about 25 percent to 43 percent – and Bedi wants that number to keep rising. “Our IT team now has the time to experiment with how the rest of the organization can benefit from intelligent automation applied to their business processes,” Bedi notes.
You’ll also need to audit IT workflows, identify the success metrics for your work, and help others in the organization help themselves using automation. For detailed advice, see the full article: Getting started with automation: 6 tips.
What best practices should we use for IT automation?
As Bedi’s experience shows, automation success is far from automatic. Also, if you focus just on tools, you’ll soon see that people and process are key.
Here are some best practices to keep in mind:
Get buy-in from your team: Of course, this advice applies to any significant shift in IT strategy, but it’s particularly true with automation, which sometimes carries negative connotations, notes Ned Bellavance, director of cloud strategy at Anexinet. As an IT leader, you’ll need to explain how the automation strategy will benefit IT and the greater business, both at an organizational and individual job level.
“Although you may already have some tools in mind, listen closely to what your team suggests. They are the ones who will be using it the most, and they won’t use it if they don't believe it can do the job,” Bellavance says. “Some other team members may have misgivings about automation and the amount of work it will take. Put their fears to rest by identifying and making some quick wins that simplify the team’s life.”
Choose flexible tools: “When selecting an ideal automation toolbox, I would focus on three key attributes: flexibility, simplicity, usability,” Bellavance advises. “Any automation tool should be flexible enough to cover 90 percent of your use cases.”
Think long-term and begin incrementally: Don't try to tackle your entire software development life cycle (SDLC) or a similarly large workflow all at once.
Ellucian CIO Lee Congdon, following this strategy, has taken a measured approach to automation in his organization. As he writes, “For us, the first step in automation is thinking about what we can consume as a service. As we rely more and more on partners to do the low-level and routine work, we can also rely on them to automate that work for us, rather than do it ourselves.
Secondly, as we shift more of these technology tasks to cloud providers, and we gain more time to work more closely with the business, we’re naturally seeing more business automation opportunities. We can see where business people are doing manual and repetitive tasks and find new ways to assist them with tools for automating processes.” (See the full article, Ellucian CIO: How we took a measured approach to automation.)
Consider a dedicated automation lead on the IT team: They will champion efforts and document success over the long haul.
Insist on strong, clear documentation for automated processes: You don’t want Rube Goldberg projects that are complex masquerading as simple.
Don’t be afraid to pass on automation for projects where it doesn’t fit. Possible example: Services that require significant customization or one-off deliveries, whether to different business units, partners, or external customers.
[ How can automation free up more staff time for innovation? Get the free eBook: Managing IT with Automation. ]
How can I make the case for IT automation within my organization?
Automation sounds scary to some people, since it is often mentally associated with job loss, and tied to some lingering misconceptions. Helping people understand the what, why, and how of your company’s automation strategy is a necessary step to achieving your goals.
We asked a variety of IT leaders for their advice on making the case for automation in your organization - up and down the ladder. Here are some tips:
Show people what’s in it for them. Show people how your automation strategy will benefit them and their jobs. Will automating a particular process in the software pipeline mean fewer middle-of-the-night calls for team members? Will it enable some people to dump low-skill, manual tasks in favor of more strategic, higher-order work – the sort that helps them take the next step in their career?
“Convey what’s in it for them, and how it will benefit clients and the whole company,” advises Vipul Nagrath, global CIO at ADP. “Compare the current state to a brighter future state, where the company enjoys greater stability, agility, efficiency, and security.”
Paint a before-and-after picture to help people see the upside, he advises.
“You want to paint a picture of the current state that people can relate to,” Nagrath says. “Present what’s working, but also highlight what’s causing teams to be less than agile.” Then explain how automating certain processes will improve that current state.
Connect automation to specific business goals. “The case for automation needs to be driven by a business demand signal, such as revenue or operating expense,” says David Emerson, VP and deputy CISO at Cyxtera. “No automation endeavor is self-justifying, and no technical feat, generally, should be a means unto itself, unless it’s a core competency of the company.”
Break the plan into manageable pieces. This will help people digest it - and reduce skepticism. It will also give you some flexibility to tweak plans as you go.
Promote your success. Talk up your small wins. Eric Kaplan, CTO at AHEAD, says that the value “small” wins reveal can actually help you sharpen the big picture for people. Kaplan points to the value of individual and organizational time as an area everyone can connect with easily.
“The best place to do this is where you can show savings in terms of time,” Kaplan says. “If we can accelerate the time it takes for the business to get what it needs, it will silence the skeptics.” (For more, see the full article: IT automation: How to make the case. )
What are some common mistakes with IT automation?
Like any IT effort, automation has its “oops” moments. Thankfully, use of automation has progressed to a point where you can learn plenty from the earlier mistakes of other people. Here are some common trip-ups to watch out for and avoid, as you automate more.
Botching your estimate of automation results: Vipul Nagrath, global CIO at ADP, says skimping on due diligence is a key automation pitfall to avoid. This is particularly true, Nagrath says, when it comes to defining the goals and expected outcomes of your automation plan.
“Be realistic. If you overestimate the benefits, you risk not achieving your stated goals,” Nagrath explains, adding that there’s a flip side: “If you underestimate the benefits, you end up underselling the program, and that can cause ‘analysis paralysis.’ Carefully estimate what the automation effort is truly going to yield.”
Thinking too narrowly about the benefits: You may have a particular IT task in your sights to automate, but you need to think broadly for the business. “Probably the most common mistake I see today is myopia,” says David Emerson, VP and deputy CISO at Cyxtera. “Automation has so many benefits.”
For example, you may be able to use automation to realize peripheral benefits such as reduced compliance audit complexity, improved security posture, and fewer manual controls and processes that hamper engineering output, he says.
Expecting too much in the first few weeks: One of the most common mistakes Bruno Attore, CTO and co-founder at Uru, has seen is under-estimating the difficulty of the early stage of implementation. Expect some trial-and-error as you iterate and optimize, he stresses.
“If you are transitioning from a pure manual operations and QA process to an automated one, the first few weeks will be hard,” Attore says. “It's important not to give up and keep pushing forward. In the end, everyone needs to know that this is not a flip of a switch, but instead, a continuous process that will only get better and better.”
Other common mistakes include believing automation will fix a bad process, failing to ensure governance, and ignoring the culture change aspect. For much more detail on these issues, see our article: 8 IT automation mistakes to avoid.
What about IT automation and security?
Since you will hear security worries from others in your organization (remember the early days of cloud security worries?), it’s important to be able to articulate how your move to automation will affect your security strategy.
The good news: Automation tools and their frequent partner, containers, can actually help improve your organization’s security posture (as many organizations have found is true of cloud services).
Orchestration tools like Kubernetes not only help you manage container deployments at scale, but they also manage related security tasks. As Red Hat security strategist Kirsten Newcomer shared in a related podcast, “You really want automation, orchestration to help manage which containers should be deployed to which hosts; monitoring host capacity; container discovery – knowing which containers need to access each other; managing shared resources, and monitoring container health.”
[ Want to learn more? Read OpenShift and Kubernetes: What's the difference? Also, get the free eBook: Getting Started with Kubernetes. ]
Newcomer encourages people to think of container security as having ten layers – including both the container stack layers (such as the container host and registries) and container lifecycle issues (such as API management). (For complete details on the ten layers and how Kubernetes fits in, check out this whitepaper: Ten Layers of Container Security. )
At the same time, more organizations are baking security into each step of the development and operations pipeline (rather than bolting it on right before teams deploy code.) Some organizations call this approach DevSecOps.
Enterprises “are finding ways to move security left in their application development lifecycles,” Newcomer told us. “They’re adopting DevSecOps by integrating security practices, tooling, and automation throughout the CI/CD pipeline.” (Read the full article: Why DevSecOps matters to IT leaders. )
Containerization can also better protect against some existing threats and help you react quickly to emerging security issues.
“The good news is that most containers are stateless and replaceable, which makes it easy to roll out a newer version of the image across a deployment and improve your security posture quickly,” Bellavance says. They should also be immutable, in that they are replaced rather than changed.
What’s next in IT automation?
When you’re dealing with IT technologies that involve this much change, it’s valuable to get a peek at what’s coming in the not-so-distant future. We’re monitoring several trends that IT leaders should keep on the radar screen.
Machine learning and AI will mature and play bigger roles: When it comes to machine learning, it’s still very early days for most organizations in terms of actual implementations. However, machine learning is expected to play a significant role in the next waves of IT automation.
“With the data that is developed, automation software can make decisions that otherwise might be the responsibility of the developer,” says Mehul Amin, director of engineering for Advanced Systems Concepts, Inc. “For example, the developer builds what needs to be executed, but identifying the best system to execute the processes might be [done] by software using analytics from within the system.”
Machine learning can also do things like enable automated systems to provision additional resources when necessary to meet timelines or SLAs, or retire resources when they’re no longer needed.
“IT automation is moving towards self-learning,” says Kiran Chitturi, CTO architect at Sungard Availability Services. “Systems will be able to test and monitor themselves, enhancing business processes and software delivery.”
Chitturi points to automated testing as an example; test scripts are already in widespread adoption, but soon those automated testing processes may be more likely to learn as they go, developing, for example, wider recognition of how new code or code changes will impact production environments.
Scripting and automation tools keep evolving: The way people view and use scripting or automation tools (sometimes referred to as configuration management tools) is evolving with greater use.
“There are many processes in the data center environment that are repetitive and subject to human error, and technologies such as Ansible help to ameliorate those issues,” says Mark Abolafia, chief operating officer at DataVision. “With Ansible, one can write a specific playbook for a set of actions and input different variables such as addresses, etc., to automate long chains of process that were previously subject to human touch and longer lead times.” (Want to learn more about this aspect of Ansible? Read the related article: Tips for success when getting started with Ansible. )
Also, the tools themselves continue to become more advanced. “With advanced IT automation tools, developers will be able to build and automate workflows in less time, reducing error-prone coding,” says Amin. “These tools include pre-built, pre-tested drag-and-drop integrations, API jobs, the rich use of variables, reference functionality, and object revision history.”
Automation opens new metrics opportunities: Automation paves the way for new ways to measure IT performance. “As more and more development activities – source control, DevOps pipelines, work item tracking – move to the API-driven platforms – the opportunity and temptation to stitch these pieces of raw data together to paint the picture of your organization's efficiency increases,” says Josh Collins, VP of architecture at Janeiro Digital.
Collins thinks of this as a possible new “development organization metrics-in-a-box.”
“Whether measuring individual resources or the team in aggregate, these metrics can be powerful – but should be balanced with a heavy dose of context,” Collins says. “Use this data for high-level trends and to affirm qualitative observations – not to clinically grade your team.”
What is Robotic Process Automation (RPA)?
Bad news if you’re picturing those cute robots that now make room service deliveries in hotels: Robotic process automation (RPA) doesn't include any actual robots. The "robot" in robotic process automation is software running on a physical or virtual machine.
“RPA is a form of business process automation that allows anyone to define a set of instructions for a robot or ‘bot’ to perform,” says Aaron Bultman, director of product at Nintex. “RPA bots are capable of mimicking most human-computer interactions to carry out a ton of error-free tasks, at high volume and speed.”
RPA can automate some of the most mundane and repetitive computer-based tasks and processes in the workplace. Think copy-paste tasks and moving files from one location to another, for example.
RPA automates everyday processes that once required human action – often a great deal of it – performed in rote, time-consuming fashion. That’s also how RPA can boost efficiency for organizations.
Where exactly does RPA make sense? Think about data-intensive, rules-based, repetitive processes – in business functions such as finance, customer service, and HR, as well as in specific industries such as insurance and financial services.
Here’s an example: Think of all the processes that must be completed when a consumer returns a product. Much of the work on the company’s side can be automated with RPA.
As you can see, RPA is not just an IT story. Since the potential for RPA use cases runs far and wide in many business environments, RPA will not necessarily be the sole purview of IT – or IT talent. People in finance or customer service might be just as likely as IT pros to become RPA stars. A growing number of training options are available if you want to build up this skill set yourself: See 8 Robotic Process Automation (RPA) training and certification courses.
Want to dive deeper on RPA? Read also:
- Robotic Process Automation (RPA): 8 habits of successful teams
- Robotic Process Automation (RPA) vs. AI, explained
- How to get started with Robotic Process Automation (RPA): 4 steps
- Robotic Process Automation (RPA): 6 open source tools
Where can I learn more about IT automation?
For more detailed advice on automation, dig into these articles, papers, and Ebooks:
Ebook: The Automated Enterprise
Deep dive: Kubernetes: Everything you need to know
Article: 5 Kubernetes trends to watch in 2020
Topic page: Understanding automation
Article: Tips for success when getting started with Ansible
Paper: Ansible in depth
Article: DevOps metrics: Are you measuring what matters?
Want more wisdom like this, IT leaders? Sign up for our weekly email newsletter.