In my last blog post I talked about how Oracle Engineered Systems, by their very nature of being tightly integrated, sometimes tend to blur the boundaries of traditional team responsibilities in the IT department. This post goes one step further into the planning stages of deployment. It is aimed mostly at DBAs, but Infrastructure Architects or Network Architects who are part of the wider team might also find it useful.
So, you’ve just heard that your organization has signed up to buy an Exadata-Exalogic solution. It’s to replace the ageing hardware with end of life support contracts that currently host a group of business critical applications. As usual, since the decision to purchase has been made and the machines will arrive in six weeks, the business has set a tight 90 day deadline for all the migrations to be completed. Since you’ve had the most to do with Oracle in the past, you’ve been nominated as the Oracle Technical Contact for the implementation. Wonderful! So where to start? What do I need to know? How different is this going to be from the usual server set up in the data centre?
Better get planning. Its time for the Six Ps of Oracle Engineered Systems otherwise you’ll have a poorly performing system on your hands and a bunch of disgruntled users.
Step One - Before you start
A few weeks prior to the arrival of the Exadata and Exalogic machines you will receive 3 very important things:
- details on how to download the Exadata Database Machine documentation
- the Oracle Exadata Deployment Assistant aka OEDA
- the Exalogic Configuration Utility aka ECU
The tools take you through a guided process to input your installation preferences and your organisation’s networking details and then generate the configuration files required by the Oracle Engineer for the initial setup of the Oracle Exadata and Exalogic Engineered Systems.
Get those files right and, provided you have the networking pre-requisites in place, your team will start migrating those business critical applications the day after you power up your Oracle Engineered Systems. Get them wrong and you’ll be praying for a miracle so you can hit your 90-day deadline!
Step Two - Do as many practice runs as time permits
As soon as you have OEDA or ECU installed on your desktop or laptop start to practice. Both tools have a preview capability that provides a snapshot of what your end state configuration will look like. So you’ll be able to try out as many different configurations as you want, and fine tune them, sharing them with the wider team for review and feedback.
WARNING! The OEDA installs the Java runtime it requires but the ECU needs OpenOffice Calc pre-installed and having a compatible Microsoft Excel installed doesn’t work!! One of the most important sections that you need to get right during your planning is the network configuration. Let’s drill down into that next.
Step Three – Understand how your networks need configuring
All Exadata and Exalogic components are self-contained and inter-connected using the built-in networking infrastructure. However, you still need to enable connectivity from your corporate network for administrative and application access. Plus the racks they sit in also need access to corporate-wide network services like time synchronization (NTP) and name resolution (DNS). Note that the Exadata Database Machine has at least 3 configured networks and optionally a 4th if required for Data Guard or backup network traffic isolation.
The Infiniband network.
Local to both racks the Infiniband network default is “wide /22”. If you have worked with Oracle Real Application Clusters, think of this as the “private” network. Unless you have a really compelling reason to change the generated default settings for this network, don’t! This network is never visible to the corporate network. However, since your Exadata machine will be inter-connected to your Exalogic machine, virtual hosts on the Exalogic will be members of this private network. So to ensure there are no IP address overlaps, you still need to carve out IP address ranges for the individual host offset to leave enough room for future expansion. No cause for alarm though, you have 1000+ IP address range to carve out the ranges from! This network is bonded in active-active mode by default.
The Client Access network.
Drawing a comparison with Oracle Real Application Clusters, think of this network as the “public network” where the RAC VIPs and the SCAN VIPs are hosted. This network is allocated from your corporate IP address space and need to be reserved by your Network Administrator. Its purpose is to facilitate Exadata database access for non-Exalogic hosted applications. Any applications running on Linux guests hosted on your Exalogic will use the inter-rack high speed infiniband network, described in the previous paragraph, for database access. To provide highly available network access, this network too is typically a bonded network.
The Administration network.
IP addresses for this network are allocated from the corporate network. It is for administrative access and every component in your Exadata rack, including their respective ILOMs, are allocated an IP address from this network. This network is not bonded.
As a general rule of thumb and where possible, reserve IP address ranges so you can allocate contiguous IP addresses for all members of a given component group. For example, if you have four Exadata DB hosts, they could be allocated .1 to .4, and their respective ILOMs could be allocated .51-.54 on the same network. This ensures that the offsets are aligned. For example, .51 would be the ILOM host offset for the component with host offset .1 and .52 would be for .2.
It is important to plan and leave some headroom for possible component expansions like adding Exadata Storage Cells. The Exadata Database Machine Installation and Configuration Guide, provided as part of the Exadata documentation set, has the details on the number of IP addresses required on each network for all Exadata Machine rack types from Full to Eighth.
There are two other decisions you need to make at the same time as reserving networks for your Exadata Database Machine. After reading the previous paragraphs you will have realised that the DB hosts have two externally visible network names. The network names on the Administration network and the Client Access network. You now need to decide which network name will be the hostname for your Exadata DB hosts and which network name will host for the default gateway that facilitates network access to the rest of your corporate network, including NTP and DNS hosts.
In my next post I will continue on the same topic and discuss planning for the Exalogic machine’s network. I’ll also include some other helpful bits to assist you with completing the Exadata (OEDA) and Exalogic (ECU) deployment templates.