Well, my learning has started with the "Nutanix Bible" which contains a wealth of information around how Nutanix has been designed and other components.It took me a whole day to go through the entire content, not that I can remember all of it now. I will be going through this the second time hopefully over the next weekend.I hope it was always so easy to find good information on all the other solutions I work on. :-) .
What I wanted to do here is make an index of all the Tips that has been provided in the "Nutanix Bible" for easy reference. All the credits here are to the author of the site " Steven Poitras ", he has done an excellent job.
1) Pro tip:
For larger or distributed deployments (e.g. more than one cluster or multiple sites) it is recommended to use Prism Central to simplify operations and provide a single management UI for all clusters / sites.
2) Pro tip:
You can determine the current Prism leader by running 'curl localhost:2019/prism/leader' on any CVM.
3) Pro tip:
You can also get cluster wide upgrade status from any Nutanix CVM by running 'host_upgrade --status'. The detailed per host status is logged to ~/data/logs/host_upgrade.out on each CVM.
4) Pro tip:
For larger deployments Glance should run on at least two Acropolis Clusters per site. This will provide Image Repo HA in the case of a cluster outage and ensure the images
will always be available when not in the Image Cache.
5) Pro tip:
Check NTP if a service is seen as state 'down' in OpenStack Manager (Admin UI or CLI) even though the service is running in the OVM. Many services have a requirement for time
to be in sync between the OpenStack Controller and Acropolis OVM.
6) Pro tip:
Data resiliency state will be shown in Prism on the dashboard page.
You can also check data resiliency state via the cli:
# Node Status ncli cluster get-domain-fault-tolerance-status type=node.
# Block Status ncli cluster get-domain-fault-tolerance-status type=rackable_units.
These should always be up to date, however to refresh the data you can kick off a Curator partial scan.
7) Pro tip
You can override the default strip size (4/1 for “RF2 like” or 4/2 for “RF3 like”) via NCLI ‘ctr [create / edit] … erasure-code=<N>/<K>’ where N is the number of data blocks
and K is the number of parity blocks.
8) Pro tip
It is always recommended to have a cluster size which has at least 1 more node than the combined strip size (data + parity) to allow for rebuilding of the strips in the event
of a node failure. This eliminates any computation overhead on reads once the strips have been rebuilt (automated via Curator). For example, a 4/1 strip should have at least
6 nodes in the cluster. The previous table follows this best practice.
9) Pro tip
Erasure Coding pairs perfectly with inline compression which will add to the storage savings. I leverage inline compression + EC in my environments.
10) Pro tip
Almost always use inline compression (compression delay = 0) as it will only compress larger / sequential writes and not impact random write performance. Inline compression also pairs perfectly with erasure coding.
11) Pro tip
Use performance tier deduplication on your base images (you can manually fingerprint them using vdisk_manipulator) to take advantage of the unified cache.
Use capacity tier deduplication for P2V / V2V, when using Hyper-V since ODX does a full data copy, or when doing cross-container clones (not usually recommended as a single container is preferred).
In most other cases compression will yield the highest capacity savings and should be used instead.
12) Pro tip
Create multiple PDs for various services tiers driven by a desired RPO/RTO. For file distribution (e.g. golden images, ISOs, etc.) you can create a PD with the files to
replication.
13) Pro tip
Group dependent application or service VMs in a consistency group to ensure they are recovered in a consistent state (e.g. App and DB).
14) Pro tip
The snapshot schedule should be equal to your desired RPO.
15) Pro tip
The retention policy should equal the number of restore points required per VM/file.
16) Pro tip
Ensure the target site has ample capacity (compute/storage) to handle a full site failure. In certain cases replication/DR between racks within a single site can also make sense.
17) Pro tip
When using a remote site configured with a proxy, always utilize the cluster IP as that will always be hosted by the Prism Leader and available, even if CVM(s) go down.
18) Pro tip
Use reserve hosts when:
You have homogenous clusters (all hosts DO have the same amount of RAM)
Consolidation ratio is higher priority than performance
Use reserve segments when:
You have heterogeneous clusters (all hosts DO NOT have the same amount of RAM)
Performance is higher priority than consolidation ratio.
19) Pro tip
You can override or manually set the number of reserved failover hosts with the following ACLI command:
acli ha.update num_reserved_hosts=<NUM_RESERVED>
20) Pro tip
Keep your hosts balanced when using segment based reservation. This will give the highest utilization and ensure not too many segments are reserved.
21) Pro tip
In ideal cases the hit rates should be above 80-90%+ if the workload is read heavy for the best possible read performance.
22) Pro tip
When looking at any potential performance issues I always look at the following:
Avg. latency
Avg. op size
Avg. outstanding
For more specific details the vdisk_stats page holds a plethora of information.
23) Pro tip
If you're seeing high read latency take a look at the read source for the vDisk and take a look where the I/Os are being served from. In most cases high latency could be caused by reads coming from HDD (Estore HDD).
24) Pro tip
Random I/Os will be written to the Oplog, sequential I/Os will bypass the Oplog and be directly written to the Extent Store (Estore).
