Pages

Advertisement

Wednesday, August 28, 2019

Amazon Web Services (AWS) : EC2-User Data

 
EC2 - user data is one of the seldom used features, yet a powerful feature. 
It is basically a simple script which can is executed at boot time by the system, the script has sudo 

 It can be used for - 
• Installing updates
• Installing software
• Downloading common files from the internet

• Anything you can think of

 It is configured in Advanced section of the Instance Launch Page. 

 Below is an example of apache Service install and launch using an ec2-script. 

 #!/bin/bash yum update -y
yum install -y httpd.x86_64
systemctl start httpd.service
systemctl enable httpd.service

echo "Heloo World" > /var/www/html/index.html/index.html


The greatest of the use of EC2-user data is when using load balancing, starting parallel instances and any boot time action you want to perform (manually or automatically).

 Remember - the header #!/bin/bash is mandatory and without it your script will not work. 

Tuesday, August 27, 2019

Oracle Exadata: chrony Daemon (release 19.1)

 
Release 19.1 of Exadata has taken a leap in using chrony daemon instead of age old written NTP daemon and protocol.

 You will not find any NTP daemon installed once you upgrade to this release and note in this release you are at Linux 7. 

 To check status of chrony daemon here are few commands which will help. 

 1. Check Sources

 # chronyc sources
210 Number of sources = 3
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^? aa-bbb-ccc-1100.dddddddd>     3   4   377     3  -2328ms[-2328ms] +/- 8332ms
^? bb-bbb-ccc-1100.dddddddd>     3   4   377     4  -2358ms[-2358ms] +/- 8681ms

^? cc-bbb-ccc-1100.dddddddd>     3   4   377     7  -2321ms[-2321ms] +/- 8696ms


2. Check Source stats

# chronyc sourcestats
210 Number of sources = 3
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
aa-bbb-ccc-1100.dddddddd>   6   3    81   +951.004   4830.397  -2334ms    31ms
bb-bbb-ccc-1100.dddddddd>   9   4   129   -438.641    216.568  -2372ms  4677us
cc-bbb-ccc-1100.dddddddd>  26  11   421   -381.175     20.365  -2332ms  3727us



3. Check sync

# chronyc tracking
Reference ID    : 0A901009 (cc-bbb-ccc-1100.dddddddd)
Stratum         : 4
Ref time (UTC)  : Mon Aug 26 03:19:47 2019
System time     : 0.000000304 seconds fast of NTP time
Last offset     : -0.000190595 seconds
RMS offset      : 0.012864766 seconds
Frequency       : 84.308 ppm slow
Residual freq   : -0.004 ppm
Skew            : 0.720 ppm
Root delay      : 0.140609756 seconds
Root dispersion : 1.491145134 seconds
Update interval : 16.1 seconds
Leap status     : Normal


You can find more details on chrony commands here 

Friday, August 23, 2019

Amazon Web Services (AWS) : Budget

 
One of the best feature of learners is the AWS budget feature. 
this feature sends alerts to you when you reach a certain criteria of Budget. 
For example if you budget for say 5 dollars, and can get alerted when you reach 50% of your limit. 

 This can help reduce your spend cost, due to instances which might have been spin up and you totally forgot about it. 
You can follow the below process to create budget. 




Rest is mostly intuitive, (Most of it is intuitive by the way)

Wednesday, August 21, 2019

Oracle Exadata: How to check if Flash Cache is Write Back or Write Through

 
This is one of the questions which newbee's into exadata ask me quite often that how to know if flashcache is Write Back or Write Through.

 Well the answer is pretty simple and this is how you check it. 

 Log into cell server on which you want to know and run below 

 # cellcli -e list cell attributes flashcachemode

         WriteBack


Or if you want to see for all cells you can use dcli 

# dcli -l root -g cell_group ' cellcli -e list cell attributes flashcachemode'

myexaexcel01: WriteBack
myexaexcel02: WriteBack
myexaexcel03: WriteBack
myexaexcel04: WriteBack
myexaexcel05: WriteBack
myexaexcel06: WriteBack
myexaexcel07: WriteBack
myexaexcel08: WriteBack
myexaexcel09: WriteBack
myexaexcel10: WriteBack
myexaexcel11: WriteBack
myexaexcel12: WriteBack
myexaexcel13: WriteBack
myexaexcel14: WriteBack
myexaexcel15: WriteBack

Oracle Database: TFACTL Blackout ORA-600/7445/700

 
tfactl or oracle's trace file analyzer utility can be really helpful sometimes, and also a lot of pain in many, when it sends a lot of alerts some of which you know and it's a known issue or you don't want to do anything about it. 

 Fortunately, you can blackout such events from tfactl 

 To supppress a specific string ORA-0600 this is what I would do. 
tfactl blackout add -targettype database -target mydb -c -event  "kewrspbr_2"  -timeout none -reason "skipping ORA-0600 Doc ID 2313603.1" 

 where 
-target : dbname
-c : clusterwide
-event: String contained in the event (find it from the email you got)

-timeout : no timeout

 Similarly, you can play around with the parameters and blackout a lot of unwanted emails.
You can find them documentation below. 
Reference - https://docs.oracle.com/en/engineered-systems/health-diagnostics/trace-file-analyzer/tfaug/running-administration-commands.html#GUID-56A03EC7-3324-48BD-9701-2411626D16CB

Tuesday, August 20, 2019

Oracle Enterprise Manager: How to Supress 'EM Event: Warning: - User SYS logged on '

 
User Logged on can be really pain as it makes a lot of noise, It is one of the collected metrics without any threshold and will generate a warning or critical message (and probably clear as well) as per your configuration.

 so, what do you do about it ? 

 There can be many solutions
1. Remove SYS from Auditing 

  • Go to database home page
  • Oracle Database (drop down menu)/Monitoring/Metric and Collection Settings
  • Scroll down to:   User Audit/Audited User
  • make sure that no values are set for the 'warning' and 'critical' boxes apply the changes 
2. Remove this from Template if you want to do it for all Databases

3. The final solution is that the metric is still collected and you see it in the database, however you just remove it from the Incident rules and notification.
For this first, identify the Incident rule which created this, you will find this in the alert email. 
  • Go to Incidents -> incident rules
  • Select your In
  • Edit the incident rule 
  • and then in the filter 'Select a Filter' Event not equals 'UserAudit:username'cident rule



This will stop the notification emails for user log-ons all together.

Oracle Enterprise Manager : Internal error detected: oracle.sysman.gcagent.task.TaskZombieException

 
I was getting a lot of error messages/incidents and alerts stating 'Internal error detected: oracle.sysman.gcagent.task.TaskZombieException'
in my Enterprise Manager (Release 13cR2)

 The right way should be to find out what is causing these (generally failed / timed out responses)
However, they are also caused due to bugs in OEM.

 The way to suppress these is as below 

 # cd $AGENT_HOME/agent_inst

 # bin/emctl stop agent
# cd sysman/config
# cp emd.properties emd.properties.bkp_19Aug2019
# vi emd.properties

 Append Below 

 _zombieSuspensions=true
_canceledThreadWait=900
_zombieThreadPercentThreshold=0
_zombieCreateIncident=false
_cancelThreadTimeout=300
_traceSeverityForTypes=noTypes

 # cp s_jvm_options.opt s_jvm_options.opt.bkp_19Aug2019
# vi s_jvm_options.opt

 Append Below at EOF

 -XX:+UseMembar
# $AGENT_HOME/agent_inst/bin/emctl start agent

this will stop creating incidents for your zombie processes. 
Reference - MoS - Doc ID 1468586.1 

Oracle Exadata: How I removed specific Alerts (Diagnostic Package cannot be uploaded to ASR server)

 
On a weekly basis, I was getting emails (after patching to release 18) that Diagnostic Package. 

 I then checked with Oracle about this and found out this is a new feature (12 release) and it is okay to ignore. But I was really bugged by the nuisance emails. So I thought of clearing it.

 But I have more than 5 Exadatas which are full racks and I do not want to clear all alerts but these itself (for Diagnostic Package)

 So, I came up with a small automation which will generate the right commands for me.

 Here is what I did 

 For Cell Servers 
# dcli -l root -g cell_group 'cellcli -e list alerthistory' |  grep 'Diagnostic packages' | awk '{print "dcli -l root -c " $1 " \x27" "cellcli -e drop alerthistory "  $2 "\x27" "   ### " $3}' | sed 's/://'

 dcli -l root -c myexacel01 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:16:52-04:00
dcli -l root -c myexacel02 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:19:15-04:00
dcli -l root -c myexacel03 'cellcli -e drop alerthistory 5_1'   ### 2018-10-06T19:18:51-04:00
dcli -l root -c myexacel04 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:17:49-04:00
dcli -l root -c myexacel05 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:18:55-04:00
dcli -l root -c myexacel06 'cellcli -e drop alerthistory 5_1'   ### 2018-10-06T19:18:46-04:00
dcli -l root -c myexacel07 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:18:13-04:00
dcli -l root -c myexacel08 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:19:01-04:00
dcli -l root -c myexacel09 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:18:36-04:00
dcli -l root -c myexacel10 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:19:26-04:00
dcli -l root -c myexacel11 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:18:16-04:00
dcli -l root -c myexacel12 'cellcli -e drop alerthistory 5_1'   ### 2018-10-06T19:18:11-04:00
dcli -l root -c myexacel13 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:17:53-04:00

dcli -l root -c myexacel14 'cellcli -e drop alerthistory 4_1'   ### 2018-10-06T19:18:03-04:00

 For DB Servers 
# dcli -l root -g dbs_group 'dbmcli -e list alerthistory' |  grep 'Diagnostic packages' | awk '{print "dcli -l root -c " $1 " \x27" "dbmcli -e drop alerthistory "  $2 "\x27" "   ### " $3}' | sed 's/://'


dcli -l root -c mydbnode01 'dbmcli -e drop alerthistory 7_1'   ### 2019-08-14T08:06:48-04:00
dcli -l root -c mydbnode02 'dbmcli -e drop alerthistory 7_1'   ### 2019-08-14T08:06:42-04:00
dcli -l root -c mydbnode03 'dbmcli -e drop alerthistory 7_1'   ### 2019-08-14T08:06:50-04:00
dcli -l root -c mydbnode04 'dbmcli -e drop alerthistory 7_1'   ### 2019-08-14T08:06:38-04:00
dcli -l root -c mydbnode05 'dbmcli -e drop alerthistory 7_1'   ### 2019-08-14T08:06:43-04:00

Friday, August 16, 2019

Oracle Exadata: DBM-02643: DROP ALERTHISTORY command did not include all members of the alert sequence for *_*

 
In this blog I am going to discuss an error message which you might get when dropping individual alerts via dbmcli or cellcli in Exadata

 if you try to drop your alert using a command like below 
drop alerthistory  2_1, 2_2, 2_3, 2_4, 2_5, 2_6, 2_7, 2_8, 2_9, 2_10, 2_11, 2_12, 2_13

 and you get an error message which is something like 

 DBM-02643: DROP ALERTHISTORY command did not include all members of the alert sequence for 2_1. All members of the sequence must be dropped together

 That means you have not selected all the 2_* members, so, in order to proceed put all the 2_* members or the sequence you are trying to drop. 

 This will clear the alert.

Friday, August 9, 2019

Oracle Enterprise Manager : java.lang.OutOfMemoryError: Java heap space

 
If you are facing a lot of issues with Agent crashing in EM 12c/13C then probably you are running out of memory. 

 An issue which is prominently seen on servers with high number of targets (50 or more)
the default configuration of Xmx and Permgen is not enough. 

 You must increase it. 

To increase you must 
1. Backup the existing configuration
# $AGENT_HOME/agent_inst/sysman/config
# cp emd.properties emd.properties.bkp

 2. Stop agent
# $AGENT_HOME/agent_inst/bin/emctl stop agent 

 3. increase the Xmx and Permgen
# $AGENT_HOME/agent_inst/bin/emctl setproperty agent -name 'agentJavaDefines' -value '-Xmx1024M -XX:MaxPermSize=192M'

 4. Start the Agent
# $AGENT_HOME/agent_inst/bin/emctl start agent 

Thursday, August 8, 2019

Oracle Exadata: How to find Serial # (to Put in Service Request)

 
In this blog I am going to talk about, how to find out the serial number of Exadata. 
it is a single command, which can be run from any node (compute or storage) using root user , the output will give you the serial number
# ipmitool sunoem cli 'show System' | grep -i Serial

        serial_number = AK00418863

You can put this serial number when raising SR for an engineering system. (Assuming the CSI is already added into your Support account)

Wednesday, August 7, 2019

Oracle Exadata : AIDE (Advanced Intrusion Detection System)

 
AIDE (Advanced Intrusion Detection System) is basically a utility which keeps the database of all file system files on the server and  alerts you via e-mail (as configured) what files and their content have changed. 
(AIDE was released with Image release 19 of Exadata)

 This can help in detecting any security threats. 

 It is a daily cron job which runs, scans and alerts

 To know if it is enabled 
$  /opt/oracle.SupportTools/exadataAIDE -status

AIDE: daily cron is currently enabled.

 If you are getting alerts and have changed the FS and are happy with the log in '/var/log/aide/aide.log'
then you can update the AIDE database which will ensure alerts are not sent to you.

 $  /opt/oracle.SupportTools/exadataAIDE -update

Tuesday, August 6, 2019

Oracle Exadata / ODA : How to find which version of engineered system you are running

 
A common question which new comers ask me is how to find out which version of  Engineered system are we running.
The answer is easy and straight forward. Just one command run as root on compute node or Storage Node (As root)

 # ODA Example  $  ipmitool sunoem cli 'show System' | grep model
         model = ODA X7-2M
        component_model = ORACLE SERVER X7-2

# Exadata Example  $  ipmitool sunoem cli 'show System' | grep model
        model = Exadata X7-8
        component_model = ORACLE SERVER X7-8

Friday, August 2, 2019

Oracle Database: Permissions to Execute Job Class

 
There is a great mis conception in DBA's who primarily work on OLTP about Job classes.

 Job classes are one of the essentially used components in job definitions which people seldom use and seldom know about. 

 This blog however is a small and nice one which talks about how to see permissions to execute a job class.

 Yes you read that right. You need execute permission for a job class. 
Only if you have execute permission to job class, you will be able to add it to your create job. 

 the way to grant is same 

 Grant execute 'on job class' to <user>';

 to verify you should use dba_tab_privs to know if your user has execute privilege on job class or not. 

 select * from dba_tab_privs where grantee = '<your user>' and type = 'JOB CLASS';

 Example output below (from TOAD)

By default permission to execute 'DEFAULT JOB CLASS' is granted to PUBLIC, so you if you do not specify any job class name, default job class will be taken and it will be okay. 

Popular Posts

© Copyright 2019 DBA to DA. . Powered by Blogger.