RAC

The CRS has four components:
    OPROCd - Process Monitor Daemon
    CRSd - CRS daemon, the failure of this daemon results in a node being reboot to avoid data corruption
    OCSSd - Oracle Cluster Synchronization Service Daemon (updates the registry)
    EVMd - Event Volume Manager Daemon

OPROCd - provides the I/O fencing for the Oracle cluster using hangcheck or watchdog timer.
CRSd -  manages starting and stopping the services and failover of the application resources, manages the OCR and stores the current know state of the cluster.
OCSSd - provides basic Group Services Support
EVMd - processes called evmlogger generates the events when things happen

Oracle Cluster Ready Services (CRS) uses the "Oracle Cluster Registry (OCR)" to keep the cluster configuration.
OCR keeps details of all resources and services, stores name and value pairs of information such as resources used to manage CRS Stacks.
OCR is loaded as cache on each node and update the cache. Only Master node write cache to OCR file.

voting disk is shared by all nodes within the cluster and information about the cluster is constantly being written to the disk, this is know as the heartbeat

Global Resource Directory (GRD) - All the resources in the cluster group form a central repository. Each instance masters some set of resources and together all instances form the GRD. The GRD is managed by two services called Global Caches Services (GCS) and Global Enqueue Services (GES), together they form and manage the GRD

Process:
LMSn - Lock Manager Server process - GCS  handles the consistent copies of blocks that are transferred between instances, rolls back any uncommitted transactions, handles global deadlock detection and monitors for lock conversion timeouts.
LMON - Lock Monitor Process - GES  maintains consistency of GCS memory structure in case of process death, responsible for cluster reconfiguration and locks reconfiguration
LMD - Lock Manager Daemon - GES manages the enqueue manager service requests for the GCS
LCK0 - Lock Process - GES  builds a list of invalid lock elements and validates lock elements during recovery
DIAG - Diagnostic Daemon  - monitor the health of the cluster

Unique parameters:
instance_name - name of the Oracle instance(default is the value of the oracle_sid variable)
instance_number - unique number greater than 0
thread - set of redolog files to be used
undo_tablespace - name of the undo tablespace
rollback_segments - use Automatic Undo Management
cluster_interconnects - use if only if Oracle has trouble not picking the correct interconnects

cluster_database - options are true or false
active_instance_count - used for primary/secondary RAC environments
cluster_database_instances - number of instances that will be accessing the database
dml_locks - number of DML locks for a particular instance
gc_files_to_locks - number of global locks to a data file, changing this disables the Cache Fusion
max_commit_propagation_delay - synchronize the SCN
instance_groups - parallel query execution groups and assigns the current instance
parallel_instance_group - group of instances to be used for parallel query execution
gcs_server_processes - number of lock manager server (LMS) background processes used by the instance for Cache Fusion
remote_listener - register the instance with listeners on remote nodes

srvctl [start|stop] database -d <database> -i <instance>,<instance> -o [force|open|mount|nomount]