Bash101: Convert colon-delimited string into an array

A common bash scripting task that I face is how to convert a non-space or non-tab delimited string into an array. The challenge is that when you initialize an array with a set of values, the values are typically delimited by spaces or tabs. However, many of the inputs that I have to parse for storing in an array are colon, semi-colon or pipe delimited. For example, in my Zone Manager script, I like to pass multivalued strings into parameters. For example, the following -S flag passes a pipe delimited string.

-S "ssh|cron|syslogd|svc:/network/ftp:default"

Here is one way that I can store that string into an array.

OPTARG="ssh|cron|syslogd|svc:/network/ftp:default"
read -r -a services <<< $(echo ${OPTARG} | sed \
   -e "s/  /_TAB_/g" \
   -e "s/ /_SPC_/g" \
   -e "s/|/ /g")
for (( i=0; i< ${#services[*]}; i++ ))
do
   services[${i}]=$(echo ${services[${i}]} | \

      sed -e "s/_SPC_/ /g" -e "s/_TAB_/      /g")
done

Although my example string didn't include any spaces or tabs, it could have.  Thus, in this example I converted spaces to "_SPC_" and tabs to "_TAB_" and then converted the pipe symbol (|) to spaces.  That way the array will be parsed according to its native delimiter, the space and properly store each value into an array entry.  Then after the array is populated with the right fields, I go back and revers the space and tab conversion to restore their values to their original state.  Note that you can use any unique string that you like for the temporary replacement values of the space and tab characters.  I just used _SPC_ and _TAB_ for simplicity.

Have a great day!

Brad

Bash101: Passing an array to a function

One of the things in bash shell scripting that often need to do but forget how to do is pass an array to a function. The following is a reference to help me remember how its done.

#!/usr/bin/bash
array1=('1' '2' '3 4 5')
show_array() {
   array2=( "$@" )
   for (( i=0; i< ${#array2[*]}; i++ ))
   do
      echo "$i: ${array2[${i}]}"
   done
}
show_array "${array1[@]}"

Have a great day!

Brad

Identity Theft Lurkes In Your Copy/Printer Room!

One of the elders of my church recently brought to my attention a CBS special titled Digital Photocopiers Loaded With Secrets You can watch the video below.



When I thought about it for a second, it was was obvious. Any device that temporarily stores data on a hard drive or any storage media for that matter will have some residue left behind after the file is removed. On most modern copiers and scanners, files are temporarily stored or queued to a storage device such as a hard drive or flash drive. Once the process in progress completes (e.g. the file is printed) the queue manager will delete the file. However, in most cases the file isn't securely deleted. It is actually just marked as deleted so that the filesystem can reclaim the space occupied by file.

The Problem
Most filesystems handle file deletion by marking the file as deleted but don't by default delete the contents of the file. This is intended to make data recovery easier so that if the drive starts to go bad or you accidentally delete a file you can still retrieve its contents. There are many freely available tools to "Undelete" files from a windows NTFS or FAT filesystem. And before anyone (like me) starts to assert that Linux and Mac/OSX filesystems don't have that problem, they are susceptible to this problem as well. Basically, if data has been written to the storage device there are tools out there that can be used to recover some percentage of the data that has been "Deleted".

The Solution
So what is a person or business to do to solve this problem. Fortunately, there are many possible solution sets to help address this problem. Like most technology related problems the solution sets fall into three categories: People, process and technology ... in that order.

People
One of the best solutions is to educate the people in your organization on how to handle confidential information. For example, printing off your customer address book at the local copy store probably isn't the best way to protect your customer's identity data.

Process
Be sure to have processes and procedures focused on secure printing and photocopying practices. In some cases, this means that security sensitive information never leaves the work premises whether in electronic or printed form. In others it may mean that all security sensitive information is only printed on the printers with encrypted drives. In others it may mean that no one can photo copy security sensitive information on public copiers.

Technology
Even though people and process are a huge component of any solution, most people run to technology to solve these sorts of problems. Fortunately, technology can help. Here are a few pieces of technology to consider.

Encrypt The Data
Consider purchasing the encryption option for your printer or copier. This option encrypts the data before it is stored on the storage device. If the device is ever stolen or copied, the probability of someone recovering the data from the device is very low. As the CBS video points out this option may tack on another $500 to your printer or copier. However, consider the risks that you run if the data contained on the storage was ever compromised.

Encryption can also be applied to the desktop as well. There are many solutions for Windows, OSX, and Linux for encrypting the full disk drive or a partition of your hard drive to ensure that the drive is ever stole, the data cannot be recovered. TrueCrypt is a great open source and cross platform solution that works very well for this purpose.

Cleanse The Media
This is actually something I personally believe that every person and business should institute as a standard operating procedure when retiring any and all storage media. To cleanse media is to wipe the contents off in such a way that it can never be recovered. My favorite tool of choice is Darik's Boot and Nuke (a.k.a. DBAN). With DBAN, you can apply military grade cleansing to just about any disk media. Disposing of other media should be done secure means as well. For example, shred DVDs, CDs and tape with a high grade cross-shredding device. Don't just throw them away.

Control Access
Probably the simplest thing that most people can do is to control access to your printing and copier devices by putting them in a locked room. Don't let people use those devices unless they have permissions to use them. Most business class photocopiers and printers also support electronic control as well by requiring a user to login before they can print or copy a document.

Note that access control also applies to the electronic domain as well. Companies would be wise to apply secure access controls to internal web sites, file shares, and backup repositories. Oracle offers a full platform for enterprise wide and web access controls, single signon and federation through their identity management suite.

Apply Information Rights Management
A very cool Oracle technology that I learned about during the transition from Sun to Oracle is in the area of Information Rights Management. In a nutshell, IRM controls the content that your employees interact with. It can control what they can read, what they can print, who they can share it with and so on. I'm not a control freak but I have worked with many customers that are and this is a great product for them. Click here watch a brief video on Oracle's IRM solution.

Dispose Securely
When disposing of printers, copiers, computers or anything that might have a storage device, be sure to wipe the data before getting rid of the item. If you don't know how to wipe the device yourself, find someone that you trust that will do it for you. This goes for papers as well. Have a locked storage bin that people can easily throw away security sensitive information into. Then securely shred the contents of those bins on a regular basis.

I hope that you find this information helpful! If so, pass it along to your friends so that we can all be a little more secure.

Brad

PS: As a disclaimer, I am an employee of Oracle. However, I would have written this blog post even if I wasn't an Oracle employee because security is everyone's problem.

ODSEE Entry Counting Methods

By way of follow up to my CPU counting blog post last week, this week I will show you a couple of ways in which to determine the number of entries that exist within an Oracle Directory Server Enterprise Edition (a.k.a. ODSEE) directory server.

There are four methods for determining the number of entries that a directory server contains.

1. Entry count by errors log.
The first is the simplest and does not require privleged access to the server.  By this method, you simply find the line(s) containing the entry count informational message within the errors log file(s). The following command line syntax where "/path_to_logs/" is the path to the errors log file should work for DSEE 6 and 7.

# egrep -i "starting up|in the directory" /path_to_logs/errors*

This should produce one or more lines that include information similar to the following:

INFO: 99966 entries in the directory database.

2. Entry count by ldapsearch.

The second method simply retrieves the entry count per suffix over the LDAP protocol.  Once you have the entry count for all suffixes on a particular server, you can sum the counts to get the full entry count for a given server.

The first step in this process is to determine what backend suffixes exist on the directory server with the following command:

# ldapsearch -h -p -D -w -s sub \
   -b "cn=ldbm database,cn=plugins,cn=config" \
   "objectclass=nsBackendInstance" cn

The resulting output may look similar to the following:

version: 1
dn: cn=example,cn=ldbm database,cn=plugins,cn=config
cn: example

dn: cn=People,cn=ldbm database,cn=plugins,cn=config
cn: People

From this example, the two backend suffixes for this directory server are "example" and "People".

The second step is to get the entry count per backend suffix.  For example, I used the following command to retrieve the entry count for the People suffix.

./ldapsearch -h -p -D -w \
   -b "cn=monitor,cn=People,cn=ldbm database,cn=plugins,cn=config" \
   "objectclass=*" ldapentrycount
The resulting output should look similar to the following:

version: 1
dn: cn=monitor,cn=People,cn=ldbm database,cn=plugins,cn=config
ldapentrycount: 3084
Repeat the above steps for all suffixes and sum them for the total of entries contained on a specific directory server.

Note that most directory server deployments have a single suffix that is replicated to all servers in the topology.  So in most cases you shouldn't have to sum more than one or two suffixes.

For more information on the entry count by ldapsearch method, consult the SunSolve document 1-9-72335.

3. Entry count by dn count.

The dn count method simply tabulates the entry count by counting the number of dn lines from an export from the directory server.  The prerequisite for this method is that you must have first created an export from the directory server instance by creating an export or performing a search for all entries.  The latter is never preferred and may fail because of denial of service controls implemented in the directory server or directory proxy server.  For example, the following search limits from the admin guide can be applied.

• The look-through limit specifies the maximum number of entries examined for a search operation.
  
• The size limit specifies the maximum number of entries returned in response to a search operation.
  
• The time limit specifies the maximum time spent processing a search operation.
  
• The idle timeout specifies the maximum time a client connection can remain idle before the connection is dropped.

Once the ldif file is produced, you can simply tabulate the entry count with the grep command like in the following example: # grep -c "^dn: " my_ldif_export.ldif 100008

4. Entry count by Console.

The console method simply brings up the console and looks up the number of entries per suffix and sums the total per server.  I will add screen shots for the DSEE Directory Server Control Center web console later.

For 5.x versions of the Sun/iPlanet/SunOne/Sun Java System Directory Server, you can lookup the entry count per suffix through the Java Console via the following instructions. 4.2.1 Bring up the Directory Server admin console and double click on one of the Directory Server master-suppliers.

4.2.2 Select the “Directory” tab and open the tree in the left pane “cn=config” -> “plugins” -> “ldbm database”. The various suffixes will be listed here. Single click on a suffix – for example “testdata” as shown below. This step and the following step will need to be done for each suffix.

4.2.3 Double click on the “monitor” object in the right pane. An example of the “monitor” object is shown below. The number of entries contained within this suffix are maintained in the “ldapentrycount” attribute. Adding up the values for each “ldapentrycount” in each “monitor” object of each suffix will provide the total entry count for a particular Directory Server master-supplier.

That is it.  I hope you find this helpful. Have a great day! Brad PS: Thanks to Bob Faller for the 5.x instructions and screen shots.

Brad

How many CPUs does server X have?

One of the most frequently asked questions that I now get as an Oracle Principal Sales Consultant is "How many CPU's does server X have?" The reason for the question is that Oracle largely prices its software by CPU count verses other methods.

Toward that end, I put together this reference for anyone that needs this kind of information.

Note that their are typically multiple methods by which to determine the number of CPUs. Further, some commands show just the number of physical CPUs while others list the the number of threads, or cores, or Hyper-Threads. With that said, here are the methods per operating system.

---

Solaris

The following command will give you the total number of physical processors in the server.
# psrinfo -p
2

The following command will return the total number of threads/cores in the server.
# psrinfo | wc -l
8

The following command will return detailed information about each physical processor.
# psrinfo -pv
2

The physical processor has 4 virtual processors (0 2 4 6)
x86 (chipid 0x0 GenuineIntel family 6 model 15 step 7 clock 2333 MHz)
Intel(r) Xeon(r) CPU E5345 @ 2.33GHz
The physical processor has 4 virtual processors (1 3 5 7)
x86 (chipid 0x1 GenuineIntel family 6 model 15 step 7 clock 2333 MHz)
Intel(r) Xeon(r) CPU E5345 @ 2.33GHz

---

Linux

The following command will return the total number of threads/cores in the server.

# grep -c "^processor" /proc/cpuinfo

2

The following command will return detailed information about each physical processor.

# egrep -i "^model name|^core id|^cpu MHz" /proc/cpuinfo

model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4200+

cpu MHz : 1000.000

core id : 0

model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4200+

cpu MHz : 1000.000

core id : 1

---

HP-UX

The following commands will return the total number of threads/cores in the server.

# mpsched -s | grep "Processor Count"

Processor Count : 4

or

# machinfo | grep "Number of CPUs"

Number of CPUs = 4

or

# /usr/sbin/ioscan -kf | grep processor | wc -l

4

The following command will return detailed information about each physical processor.

# machinfo

CPU info:

Number of CPUs = 4

Clock speed = 1595 MHz

Bus speed = 532 MT/s

CPUID registers

vendor information = "GenuineIntel"

processor serial number = 0x0000000000000000

processor version info = 0x0000000020000704

Reference:

* Howto find CPU is dual/quad core in HP-UX

---

AIX

The following commands will give you the total number of physical processors in the server.

# lsdev -Cc processor |wc –l

or

# prtconf | grep Processor

or

The following command will return the total number of threads/cores in the server.

# bindprocessor -q

The following command will return detailed information about each physical processor.

# prtconf

and

# lsattr -El sys0 -a modelname

and

# lparstat -i|grep ^Active\ Phys

---

Windows

Use the following sequence of mouse clicks to view the CPU information for the server.

1. Right click the button on the "Start" icon in the lower left hand corner of the screen,

2. click on "Explore".

3. Right click on "My Computer"

4. and click on "Properties".

You will see the type and number of CPUs under the Computer portion of the System Properties window.

Have a great day!

Brad

Oracle Open Office, the final link to freedom...

For over the last decade, my primary desktop operating system has been some distribution of Linux or Unix. In 2008, I transitioned to Apple's Unix based OSX for its seamless integration of calendar, address book, and email. In both cases, finding a good inexpensive (e.g. FREE) office suite was not easy... especially in the early years.

In 1999 Sun purchased StarDivision for their StarOffice product with the intent of open sourcing it (e.g. OpenOffice) and offering it as a commercial offering called StarOffice. For me, OpenOffice was the final link that set me free from the Microsoft Windows desktop operating system so that I could run exclusively on a Unix desktop. It was quite liberating. One of the key ingredients to OpenOffice's and its newly owned commercial counterpart (Oracle Open Office's)
success is compatibility with Microsoft Office. This means that I can open any version of Microsoft Office documents with OpenOffice/Oracle Open Office and save them again in a compatible Microsoft Office format.

If you are looking for a free office suite, give OpenOffice a try. Or, if you just want a lower cost commercial alternative to Microsoft Office, give Oracle Open Office a try.

Enjoy!

Brad

Disclaimer: I am an employee of Oracle. However, the opinions expressed in this post is my own.

Are You A Master Cloud Operative?

I challenge you to the test to see if you are a Master Cloud Operative.

Enjoy!

Brad

Iron Man 2 - Oracle Style

One of the things that I have really enjoyed about the transition from Sun Microsystems to Oracle, is a real marketing budget. Take for instance the exclusive Iron Man 2 preview found here.

Sweet!

Brad