Cause of The Problem
In a database at one time 148 user-defined roles can be enabled. Enabling more than 148 user-defined roles causes ORA-28031.

Solution of The Problem
At first time you can think about max_enabled_roles parameter but max_enabled_roles parameter has nothing to do if you enable 148 user-defined roles.

In fact this parameter is deprecated. It is retained for backward compatibility only. You can see this parameter settings by,

SQL> show parameter max_enable

NAME TYPE VALUE
———————————— ——————————— ——————————
max_enabled_roles integer 150

Though this parameter is set to 150 and you have enabled 148 user defined roles but additional user-defined roles can’t be enabled because of the actual number of roles that users can enable is 148. So max_enabled_roles is 2 plus 148, because each user has two additional roles, PUBLIC and the user’s own role.

As there is limit to 148 so you have to disable fewer roles or drop some roles and reorganize your database role policy.

You can enable or disable roles by using the SET ROLE statement for your current session.

If you want to enable all roles except manager and clerk role then use,

SET ROLE ALL EXCEPT manager, clerk;

Remember roles listed in the EXCEPT clause must be roles granted directly to you. They cannot be roles granted to you through other roles.

But in windows there is not tail command. We can monitor the alert log within oracle itself regardless of platform. This can be archived by following steps.

1)Create an External Table to read the alert log.

column a_log new_value ALOG noprint
column value new_value bkgd_dmp noprint

select ‘alert_’||instance_name||’.log’ a_log
from v$instance;

select value
from v$parameter
where name = ‘background_dump_dest’;

create or replace directory data_dir as ‘&&bkgd_dmp’;

CREATE TABLE alert_t
(
text_line varchar2(255)
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY data_dir
ACCESS PARAMETERS
(
records delimited by newline
fields
REJECT ROWS WITH ALL NULL FIELDS
)
LOCATION
(
‘&&ALOG’
)
)
REJECT LIMIT unlimited;

2)Query from the External Table.

For example, you can then query the last 21 lines of the alert log in this manner:

select text_line
from (
select rownum rn, text_line
from alert_t
)
where rn between (select max(rn) – 20 from (select rownum rn from
alert_t))
and (select max(rn) from (select rownum rn from alert_t));

In this topic I have shown how we can take automate backup through crontab tool.

To avoid error Verify that ORACLE_HOME is set properly error like as below
Message file RMAN.msb not found
Verify that ORACLE_HOME is set properly

set proper ORACLE_HOME and ORACLE_SID in your backup file.
So I created my backup_job.sh as below.

bash-3.00$ vi backup_job.sh
#!/bin/bash
export ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1
export ORACLE_SID=dbase
/oracle/app/oracle/product/10.2.0/db_1/bin/rman target sys/a@moon:1522/dbase cmdfile=/oradata1/
backup/backup_job2.sh

If you don’t set $ORACLE_HOME and don’t give full path of rman then possibly in the output file you get rman:not found error.
/oradata1/backup/backup_job.sh: rman: not found

In the script I provided full path of rman.

The actual backup script are in backup_job2.sh and from backup_job.sh it will be called through rman.

bash-3.00$ vi backup_job2.sh
backup database format ‘/oradata1/backup/%U’;

If you don’t make these script executable or don’t change permission to 777 then you likely get error in the output file as,
sh: /oradata1/backup/backup_job.sh: cannot execute
So change the permission as

bash-3.00$ chmod 777 backup_job.sh
bash-3.00$ chmod 777 backup_job2.sh
To write contents within crontab in vi editor issue,
bash-3.00$export EDITOR=vi
bash-3.00$ crontab -e
35 06 * * * /oradata1/backup/backup_job1.sh > /oradata1/backup/backupdata.log

By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .

48 06 * * * /oradata1/backup/backup_job.sh > /oradata1/backup/backupdata.log 2>&1

Now I will discuss some aspects of cronjob.

Crontab Restrictions
-You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow.

-If that file does not exist, you can use crontab if your name does not appear in the file /usr/lib/cron/cron.deny.

-If only cron.deny exists and is empty, all users can use crontab.

-If neither file exists, only the root user can use crontab.

-The allow/deny files consist of one user name per line.

Crontab Commands
Before invoking crontab use export EDITOR=vi ;to specify a editor to open crontab file.
Now open crontab as below
-crontab -e :Edit your crontab file, or create one if it doesn’t already exist.
-crontab -l isplay your crontab file.
-crontab -r :Remove your crontab file.
-crontab -v isplay the last time you edited your crontab file. (This option is only available on a few systems.)

Crontab syntax option
MINUTE HOUR DAY_OF_MONTH MONTH DAY_OF_WEEK USER COMMAND
for any field multiple value could assign, where they are separated by comma.

MINUTE accept value from 00 to 59
HOUR accept value from 00 to 23
DAY_OF_MONTH accept value from 01 to 31
MONTH accept value from 01 to 12
DAY_OF_WEEK accept value from 0 to 6 [0 for sunday, 1 for monday, ...]
–USER means that who will execute the command (optional/not common)
COMMAND the command for execution

# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0 1500 0 226707 0 0 0 68280 0 0 0 BMRU
lo 16436 0 13 0 0 0 13 0 0 0 LRU
vmnet 1500 0 0 0 0 0 6 0 0 0 BMRU

We can get the the elaboration of the preceding terms below.
* Name – The type of the Ethernet interface
* Mtu – The maximum size of the packets that are transmitted on the network.
* Net/Dest – The subnet address
* Address – The name of the system
* Ipkts – The number of packets received
* Ierrs – The number of received packets that have errors
* Opkts – The number of packets sent
* Oerrs – The number of sent packets that have errors
* Collis – The number of output packets that result in collision
* Queue – The number of packets that are queued

Considerations on the output
* The Queue field should have a 0 value.
* The Collis field value should not be greater than 5% of the no. of Opkts.
* If Ierrs field value is greater than 25% of the Ipkts field value the system is dropping packets, which causes a high number of input errors.

It is difficult to interpret the output of the netstat utility for a system with many network interfaces. In that case you can use,
$netstat -i eth1(interface name)
to get traffic of a particular interface.

SQL> select name from v$database;
NAME
———
DBASE

SQL> select instance from v$thread;

INSTANCE
——————————————————————————–
dbase

which indicates in the database there is only one instance that is database have a one-to-one relationship between datafiles and the instance. Oracle RAC environments, however, have a one-to-many relationship between datafiles and instances. In RAC environments multiple instances form a single database. The instances can reside on different servers which is referred as nodes in RAC environment.

• In RAC environment the set of interconnected instances that allows the servers to work together is called cluster. The physical structures that make up the database must reside on shared storage that is accessible from all servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.

• In RAC database each instance has its own memory and background processes but every instances share the same data files. So while any data need to write or retrieve they use the same datafiles but each instance use it’s own memory.

Now question is if one instance read data and modify it in the buffer cache and if another instance need that data then how oracle manages it?

• In RAC Oracle uses Cache Fusion Technology to synchronize the data stored in the buffer cache of each cluster database instance. Cache Fusion moves current data blocks (which reside in memory) between database instances, rather than having one database instance write the data blocks to disk and requiring another database instance to reread the data blocks from disk.

• When a data block is needed for one instance which resides in the buffer cache of another instance, Cache Fusion transfers the data block directly between the instances using the interconnect, enabling the Oracle RAC database to access and modify data as if the data resided in a single buffer cache.

Oracle RAC provides high availability and scalability.

If one instance fails in RAC environment still another instance can do the task and thus prevents the server from being a single point of failure.

In a RAC environment transactions are distributed over multiple instances. Thus minimize load from one instance.

• Never mix with oracle clusterware with Oracle RAC. The Oracle Clusterware is software that when installed on servers, enables the servers to be bound together to operate as if they were one server. Before you install oracle RAC you need to install clusterware. Without clusterware you can’t install oracle RAC.

2)Cluster Verification Utility (CVU)- The CVU is a command-line tool. It is very useful to check the nodes for preinstallation as well as postinstallation requirements of the cluster environment. In fact OUI runs the CVU after the Oracle Clusterware installation to verify the environment.

3)Oracle Enterprise Manager- With EM it is easy to configure RAC environments. It has both the Database Control and Grid Control graphical user interfaces (GUIs).

4)Server Control (SRVCTL)- SRVCTL is a command-line interface that you can use to manage an Oracle RAC database from a single point. Using SRVCTL you can start or stop or manage any instance on the cluster.

5)Cluster Ready Services Control (CRSCTL)- CRSCTL is a command-line tool that you can use to manage Oracle Clusterware. You can use CRSCTL to start and stop Oracle Clusterware and to determine the current status of your Oracle Clusterware installation.

6)Oracle Interface Configuration Tool (OIFCFG)- OIFCFG is a command-line tool which you can use to allocate and de-allocate network interfaces to components. You can use OIFCFG to direct components to use specific network interfaces and to retrieve component configuration information.

7)OCR Configuration Tool (OCRCONFIG)- OCRCONFIG is a command-line tool for OCR administration. You can also use the OCRCHECK and OCRDUMP utilities to troubleshoot configuration problems that affect the OCR.

•Swap space: If RAM is between 1 GB and 2 GB then make swap space to 1.5 times of the size of RAM.
If RMAN is more than 2GB then make swap space to the equal of the size of the RAM.

•Temporary space: At least 400 MB. Typically in /tmp directory.

•Processor type (CPU): Need to be certified with the version of the Oracle software being installed.

•Hard Disk Space: 1.5 GB for oracle database home directory+1.5GB for the ASM home directory+120 oracle clusterware software installation+Two Oracle Clusterware components OCR 256 MB each, or 512 MB total disk space+Three Oracle Clusterware components Voting Disk 256 MB each, or 768 MB total disk space.

•All the nodes in the cluster must have same hardware architecture. However can have machines of different speeds and size in the same cluster.

On *nix system you can check hardware components as follows.

•To determine physical RAM size, # grep MemTotal /proc/meminfo

•To determine the configured swap space, # grep SwapTotal /proc/meminfo

•To determine the amount of disk space available in the /tmp directory, # df -k /tmp

•To determine free disk space on the system, #df -h or #df -k

•To determine whether the system architecture, # grep “model name” /proc/cpuinfo

Checking the Network Requirements
Network Hardware Requirements
•One private interconnect is needed for Oracle Custerware for the use to synchronize each instance’s use of the shared resources and for Oracle RAC to interconnect to transmit data blocks that are shared between the instances. Thus each node need at least two network interface cards, or network adapters. One adapter is for the public network and the other adapter is for the private network

•The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. For example if in server1/node1, eth0 is the public interface then on server2/node2 eth0 must be the public interface.

•For the public network, each network adapter must support TCP/IP.

•For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP.

•Note UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware.

•For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. All nodes must be accessible between through private network. You can check the reachability by ping command.

Network Parameter Requirements
If NFS is used for the shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.
You can set the value by updating the /etc/fstab file on each node with an entry similar to the following,

clusternode:/vol/DATA/oradata /home/oradata/app nfs
rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0 1 2

IP Address Requirements
•You must have at least three IP addresses available for each node
1.An IP address for the public interface. Interface name should be the name of the node name.
2.An IP address for the private interface. Interface name should be hostname-priv.
3.One virtual IP address with an associated network name. Interface name should be hostname-vip.

•The VIP is on the same subnet as your public interface and it address will be not used currently in the network.

•For public and virtual addresses register with an associated network name in DNS. If you do not have an available DNS, then record the all network names and interface names in the system hosts file, /etc/hosts.

•Identify the interface names and associated IP addresses for all network adapters by running the following command on each node:

# /sbin/ifconfig

Node Time Requirements
Ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol (NTP) feature of most operating systems for this purpose.

Verifying the Installed Operating System and Software Requirements
•To determine which distribution and version of Linux is installed, run the following command as the root user:
#cat /etc/issue

•The Linux kernel is updated to fix bugs. These kernel updates are referred to as erratum kernels or errata levels. To determine if the required errata level is installed, use the following procedure as the root user:

#uname -r

•The OSDBA group (typically, dba) -This is the OS user who has SYSDBA privilege.

•The Oracle Inventory group (typically, oinstall) -This group owns all Oracle software installed on the system.

The following operating system users are required

•A user that owns the Oracle software (typically, oracle).

•An unprivileged user (for example, the nobody user on Linux systems).

Create the required groups and users
Do this steps on all nodes of your cluster.
As a root user,

/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
useradd -u 200 -g oinstall -G dba -d /home/oracle -r oracle
Change the password by,
passwd oracle

Verify the user by,
id oracle

Note that in this example in stead of using node1 use your original node name and instead of using node2 use your original node name.

Configuring SSH on All Cluster Nodes
When installing Oracle RAC on UNIX and Linux platforms, the software is installed on one node, and OUI uses secure communication to copy the software binary files to the other cluster nodes. OUI uses the Secure Shell (SSH) for the communication.

So you must have configured SSH on all nodes.

•To configure SSH, you must first create Rivest-Shamir-Adleman (RSA) keys and Digital Signature Algorithm (DSA) keys on each cluster node. After you have created the private and public keys, you copy the keys from all cluster node members into an authorized keys file that is identical on each node.

Generating RSA and DSA Keys
1)Log on as oracle user.

2)See whether .ssh directory exist or not. If does not exist then create one.
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh

3)Create the RSA-type public and private encryption keys by,
/usr/bin/ssh-keygen -t rsa
This command creates the public key in the /home/oracle/.ssh/id_rsa.pub file and the private key in the /home/oracle/.ssh/id_rsa file.

4)Create the DSA type public and private keys.
/usr/bin/ssh-keygen -t dsa
This command creates the public key in the /home/oracle/.ssh/id_dsa.pub file and the private key in the /home/oracle/.ssh/id_dsa file.

5)Repeat step 1 through 4 in all the nodes.

Adding the Keys to an Authorized Key File
1)Go to .ssh directory
$ cd ~/.ssh

2)Add the RSA and DSA keys to the authorized_keys files.
$ cat id_rsa.pub >>authorized_keys
$ cat id_dsa.pub >>authorized_keys

3)Using SCP copy the authorized_keys file to the oracle user .ssh directory on a remote node.
scp authorized_keys node2:/home/oracle/.ssh/

4)Using SSH, log in to the node where you copied the authorized_keys file, using the passphrase you created. Then change to the .ssh directory, and using the cat command, add the RSA and DSA keys for the second node to authorized_keys file.
ssh node2
Enter passphrase for key ‘/home/oracle/.ssh/id_rsa’:
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys

5)If you have more than 2 nodes in your cluster, repeat step 3 and step 4 for each node you intend to add to your cluster. Copy the most recently updated authorized_keys file to the next node, then add the public keys for that node to the authorized_keys file.

6)After updating the authorized_keys file on all nodes, use SCP to copy the complete authorized_keys file from the last node to be updated to all the other cluster nodes, overwriting the existing version on the other nodes. For example,
scp authorized_keys node1:/home/oracle/.ssh/

Configure SSH User Equivalency on Cluster Member Nodes
1)Log on as a oracle user.

2)Start the SSH agent and load the SSH keys into memory.
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add

3)Complete the SSH configuration by using the ssh command to retrieve the date on each node in the cluster.
$ ssh node1 date
$ ssh node2 date

4)shmall: Recommended value to install oracle is 2097152 or over. The default setting of this kernel parameter is in /proc/sys/kernel/shmall

5)ip_local_port_range: Recommended value to install oracle is Minimum: 1024 and Maximum: 65000 or over. The default setting of this kernel parameter is in /proc/sys/net/ipv4/ip_local_port_range

6)rmem_default: Recommended value to install oracle is 1048576 or over. The default setting of this kernel parameter is in /proc/sys/net/core/rmem_default

7)rmem_max: Recommended value to install oracle is 1048576 or over. The default setting of this kernel parameter is in /proc/sys/net/core/rmem_max

8)wmem_default: Recommended value to install oracle is 262144 or over. The default setting of this kernel parameter is in /proc/sys/net/core/wmem_default

9)wmem_max: Recommended value to install oracle is 262144 or over. The default setting of this kernel parameter is in /proc/sys/net/core/wmem_max

How to see the current value of the kernel parameters

1)semmsl, semmns, semopm, and semmni: # /sbin/sysctl -a | grep sem

2)shmall, shmmax, and shmmni: # /sbin/sysctl -a | grep shm

3)ip_local_port_range: # /sbin/sysctl -a | grep ip_local_port_range

4)rmem_default, rmem_max, wmem_default, and wmem_max: # /sbin/sysctl -a | grep net.core

How to modify the Kernel Parameter in Linux
1)Modifying /etc/sysctl.conf startup file.
With any text editor you can edit /etc/sysctl.conf file and then either put of change the entry like,
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=262144
net.core.wmem_max=262144

Or to change a certain parameter like kernel.shmmax you can insert it in the /etc/sysctl.conf like,
# echo “kernel.shmmax=2147483632″ >> /etc/sysctl.conf

Note that settings in /etc/sysctl.conf is persistent but not immediate affect. That is whenever your system will restart then it will in effect. In order the affect the setting of the value immediately use,
/sbin/sysctl -p -On Red Hat Linux system.

On SUSE linux to set persistently use
# /sbin/chkconfig boot.sysctl on

2)Modifying /proc file system.
If you wanted to dynamically alter the value of SHMMAX without rebooting the machine, you can make this change directly to the /proc file system. This command can be made permanent by putting it into the /etc/rc.local startup file:
Like,
debian:/home/Shaik# echo “6553″ > /proc/sys/fs/file-max

3) Using sysctl command

With sysctl command you can also change the value of SHMMAX:

# sysctl -w kernel.shmmax=2147483228

If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command.
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose

Checking the Hardware and Operating System Setup with CV
As an oracle user use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and OS setup:

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2

Checking the Operating System Requirements Setup with CVU
To check if your system meets the operating system requirement pre-installation tasks use the following syntax,
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database}
-osdba osdba_group -orainv orainv_group -verbose

If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom with the OSDBA dba and Oracle inventory group oinstall then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall