First week in Canada or cleaning internet cafe computers

Finally I have a chance to use a computer without the threat of spyware,adware and rootkits. I thought I could use a internet cafe a couple of blocks from where I am staying, but the last visit required 40 minutes of downloading spybot, ad-aware, firewalls and sysinternals rootkit revealer just to make the machine ok to use. It was affected with spyware and adware badly.

Anyway I chose to come into the Pythian office and use the computer I have been given for the duration of my stay as I know it is secure.
So it is Saturday morning Canadian (EDT) time just before lunchtime.

The first week has been busy, the learning curve steep, but I have jumped into handling the daily processes and alerts as a way to get up to speed as quickly as possible. Doing actual work as opposed to reading documentation and having meetings (whilst important) is a much faster way to "get a grip" on the new work environment, processes, clients etc.

The system that Pythian have for handling the large number of clients and larger number of databases, client -> databases is a one to many relationship :) is very very good.
Like any successful company, systems and processes are one of the keys to allowing the company to expand and also provide a method of storing or maintaining what is called organizational learning or knowledge. This way if someone leaves the company, the knowledge doesn't leave as well.

I won't post much in way of the new direction and database related posts until I get back to Australia.

More later, have fun.

Blogging from Canada

As part of my new role I am going to meet the Pythian team based in Ottawa, Canada. It will be great to meet the guys and girls face to face.

So at the moment I am sitting in the departure area of Vancouver Airport waiting for the connection to Ottawa. I have time to burn so I thought it might be fun to write an article from an airport internet terminal. Humour me...

I have a couple of ideas for new articles and maybe a extra direction for the site relating to ASX listed stocks.

Some ideas I want to pursue in the near future are

1. Using Analytical functions in Oracle in combination with Oracle statistical gathering to provide indepth and potential a new way to view performance metrics.
2. Use the Oracle Data Miner and prediction functions to actually produce predictive performance analysis based on the Oracle stats.
3. Review the state of SQLserver dates as stored by the datetime datatype. We have seen some weird dates which do not match what the online books (doco) say are the limits.
4. Further review and detailed outlines of installing Oracle on Ubuntu.
5. Expansion on the dead money comparsions (rent vs mortgage interest) to truly show the tax effective nature of buying and leasing for the first 7-10 years of the loan.
6. Build on my expanding knowledge of Analytics to produce some unique reports on ASX listed stocks.

I am also going to write some blog articles for Pythian on their blog once I get going. The first one relates to extracting a complete copy of the CTXSYS to faciliate moving a schema with text or domain indexes.

Plenty of work ahead and as I mentioned, the more popular a subject the more I will use that to guide the articles I write in the future.

My time is limited to the coins I have so I will sign off.

Have Fun

Paul

Google Ads and feed links

All those returning visitors would have noticed that there now google ads and some RSS feeder links.
This is an exercise in reviewing and working with the various web technologies rather than a chance to make any money. Given the site cracked 300 visitors this month (as measured by statcounter.com) you see what I mean.
I have been playing around with including google ads for a while. I might also upgrade the amazon stuff and actually start writing some reviews for the books I have read in my lifetime, although most of those books are not related to the theme of this blog.

The most popular pages of this blog are:

• Oracle on Ubuntu
• Downgrading Oracle Enterprise Edition to Standard Edition
• Comparison of Dead Money - Rent vs Mortgage Interest

Given the signals I am receiving from the market (of web surfers and search engine results), I should start writing some more about those areas. Signal to Noise ratio is a bit low at the moment...

Have Fun

Paul

Market Profile perl script

I posted ages ago on a ASX stockmarket forum that I had developed a perl script to display a market profile.
What is a market profile?

1. Here are links to the CBOT education site with another link to Market Profile 101
2. In a nutshell, a market profile seeks to show the distribution of prices intraday from tick or trade-by-trade data.
3. It is simple to draw, which allowed floor traders to draw (before portable devices)
4. It can reveal a difference trading pattern to what the standard OHLC price bar suggests.
   

I have access to historical tick data from the ASX and so was using perl to test various theories I had thought up about the market. Developing the market profile script was more of a challenge.

Anyway I still get requests for perl script via email even though that post was back in 2001.

As part of the move away from perl to SQL for other stuff, I am going to load my tick data into my OracleXE database and I should be able to rewrite the perl script as SQL then rewrite as a function.

If you want a copy of the perl script, email me. roobaron-/*?at*?/-yahoo.com. Take out the regex breaking bit.
or
Cut and paste the script from here

Have Fun

Paul

### CUT HERE ###

# Market Profile
# Paul Moen 05/12/2001
# Explanation:
# Takes csv file from Weblink and converts into market profile
# Traditionally market profile was split on 30 minute increments
# Each 1/2 hour is given a letter starting at A
# Range is determined by Price based on ASX tick movements
# i.e <> 1.00 increments 0.01
# So. each TP (time point) will be 5 ticks. eg price > $1.00 TP = 0.05
# Program logic based on manual procedure. i.e as data arrives time is checked
# and price is checked to see if resides with existing TP or needs a new TP
# Read the attached readme_market_profile.txt for more info on the program logic.


# Read csv file piped via sort (sorting on price)

# $title = ;

# Format of file is expected to be like this
# No, Date, Time, price, volume, sale, C Code
# Therefore @line[3] (4th element) is the price.

# Set variables

$tpr_flag = 0;
$last_price = 0;
$period_change_flag = 0;
$tp_list = '';
@mp_list = '';

while ($line = ) {
chop($line);
@line = split(/,/,$line); # split line into array elements.

# grab the date from 2nd element
$date = @line[1];
# Grab the time and determine what letter it is
# Note: Any time before 10am is ignored and time after 4.30pm is ignored
# Letter increment by 1/2 hour

# Now grab the price
$price = @line[3];
# We are going to calculate the tpr only once so check to see if it set already
if ($tpr_flag != 1){
# tpr is the time price range
if ($price > 1.00){
# tick is 0.01, tpr is 0.05
$tpr = 0.05;
$tpr_flag = 1;
} elsif ($price <> 0.10){
# tick is 0.005, tpr is 0.025
$tpr = 0.025;
$tpr_flag = 1;
} elseif ($price < tpr =" 0.005;" tpr_flag =" 1;" time =" @line[2];" period =" 'A';" period =" 'B';" period =" 'C';" period =" 'D';" period =" 'E';" period =" 'F';" period =" 'G';" period =" 'H';" period =" 'I';" period =" 'J';" period =" 'K';" period =" 'L';" period =" 'M';" period =" 'N/A';" period_change_flag =" 1;" period_change_flag =" 0;" last_price ="="" tp_list =" $price;" mp_length =" push(@mp_list,$tp_list);" last_price =" $price;" price_ge =" $last_price" price_le =" $last_price" the =","> or <> $price_le) {
# Price is within tpr and no change to , so no change to list
} elsif ( $price > $price_ge ) {
# price is greater than the price + tpr
$tp_list = $price_ge; # Price is in new tpr
$tp_list .= ' ';
$tp_list .= $period;
# push tp_list onto the top of mp_list (main list)
# @mp_list[length -1] becomes the last point.
$mp_length = push(@mp_list,$tp_list);
$last_price = $price;
# Calculate the timepoint range tpr
$price_ge = $price_ge + $tpr;
$price_le = $price_le - $tpr;
} elsif ($price < $price_le) { # price is less than price - tpr $tp_list = $price_le; # Price is in new tpr $tp_list .= ' '; $tp_list .= $period; # shift tp_list onto the bottom of the mp_list (main list) # @mp_list[0] becomes the start. $mp_length = shift(@mp_list,$tp_list); $last_price = $price; # Calculate the timepoint range tpr $price_ge = $price_ge + $tpr; $price_le = $price_le - $tpr; } else { print "something is wrong\n"; } } else { # Period is not equal to A need to work with list of scalars if (($price < $price_ge && $price > $price_le) {
# Price is in current tpr but the period is different, there should be
# a tp already in existence which requires an additional tp to be added.

} # period A check
} # everything else
$current_period = $period;
} # end of while
# end of market_profile.pl

### CUT HERE ###

Commentary of RBA Interest rate rise

The Reserve Bank of Australia (RBA) decided to raise its money market rate by 25 basis (0.25%) to 5.75%. It also released a media statement injunction with the announcement.

A quick summary (although you should go and read it)

1. International growth is driving strong growth in business profits, which is enabling business to invest and borrow on the strength of those increased cashflows.
2. Domestic consumers are able to borrow more as wage growth increases borrowing capacity.
3. Wage growth is driven by business expansion.
4. The RBA thinks the current borrowing is too much and/or too quick and wants to dampen the demand for credit by increasing the rate.
   

As much as the RBA would like to reduce the demand for credit, given it is backed by strong increases in cashflows for both businesses and consumers (on the back of business demand for workers) how far will it go?
If the business growth is coming a stronger export sector that sector will be getting its demand signals and its increasing cash flows from overseas consumers.
If those consumers are intermediate producers or producers of consumer goods for export, those producers will be taking their demand signals, stronger cashflows and the ability to borrow more from their overseas customers.

The RBA can't win, it can't set the Chinese interest rate or US interest rate or Japanese rate or EU rate, or stop individuals or companies fulfilling demand either domestically or internationally.
Instead it acts like a overbearing nanny, scolding businesses for borrowing to invest (in capital) so they can expand production, so they can produce and spend more.

The headlines have all been about poor old Australian consumer who has the massive mortgage and how the rate hike will make things harder and it will be for highly leverage consumers.
The consumer is collateral damage, this rate rise is aimed directly at reducing business borrowings for investment. When businesses cough, consumers catch a cold.

What the RBA has done is make it harder for Australian businesses to expand their production in the face of increased demands. This will limit the ability of those businesses to supply both domestic and international customers, therefore limiting their ability to spend but also limiting their ability to increase profits and most likely productivity.
The RBA is worried about price inflation, however by restricting the ability to increase supply, when demand is increasing, it's decision will cause prices to rise. That is what PRICE increases are signaling to businesses, INCREASE supply, INVEST in production.

The RBA like any other public institution needs to be seen to act otherwise its reason for existing risks being questioned. In increasing or decreasing the rate, the RBA changes the whole production structure within Australia, the structure it most likely can't fully comprehend.
It would be better leaving the rate stable, never again changing it. This would allow businesses to quit worrying about the affect of changes in interest rates on their profits and ability to expand and allow those businesses to make decisions based on price signals. Businesses try to achieve some stability by hedging the interest rates however most hedges are relatively short term.

If the RBA said that the rate was not going to change in 30 years, that stability would allow businesses to make much longer range decisions, enabling longer chains of production to emerge. Instead it plays around with the rate, decreasing it to increase the attractiveness of borrowing, then having remorse two or three years later and deciding to increase it again.

The joys of having a single Command and Control institution controlling the most important price signal of all i.e. the price of money.

Have Fun

Downgrading from Enterprise Edition to Standard Edition

The business decided that one of the databases that I support, that there was no requirement to run Enterprise Edition. So today and tomorrow morning early we (Junior DBA and I) are going to through the process of downgrading to Standard Edition.
Here are the steps Oracle recommends:

1. Export the whole database to file using exp or expdb, depending on the version.
2. Deinstall the Enterprise Edition using Oracle installer.
3. Reinstall the Standard Edition software using Oracle software.
4. Create a new database and import the data from export file.

What we did instead to minimize downtime to just the export and import process.

1. Install Standard Edition in a different Oracle home and different Oracle home path eg. $ORACLE_HOME becomes /oracle/product/920
2. Create a new database using dbca or from scripts. Applying all necessary patches.
3. Export the whole database from old database.
4. Shutdown old database.
5. Import into new database.
6. Switch listener to point at new database.

This has reduced the actual downtime from at least 2 hours to 20 minutes which is just the export/import process (the db is small and the box fast).

A couple of gotchas when using dbca to create the database.

1. LOG_ARCHIVE_DEST_n doesn't work for Standard Edition. This parameter was set by the dbca in its standard init.ora file. This parameter setting returns the error ORA-439 feature not enabled: Managed Standby. Use LOG_ARCHIVE_DEST instead for Standard Edition.
   
2. When using ssh -X hostname you can't use su or sudo to change to oracle. Tunneling X through ssh (the -X option) requires ssh -X oracle@hostname to get the right DISPLAY set.
   

Have Fun