2001-08

<body stylesrc="index.html" background="images/Bkgrnd02T2.gif"> <table border="0" cellpadding="6" cellspacing="0" width="810"> <tr> <td align="right" valign="top" height="1" width="161"> <p align="center"><img border="0" src="images/Accent03.gif" align="left" width="142" height="37"> </p> <p align="left"><b><font size="2">August 2001 - Vol. 2 No. 8</font></b></p> </td> <td height="1" width="621"> <p align="center">  <a href="index.html" target="_top">[Home]</a> <a href="Newsletters.html" target="_top">[Up]</a> <a href="2001-09.html" target="_top">[Next]</a> <a href="2001-07.html" target="_top">[Previous]</a></td> </tr> <tr> <td valign="top" height="529" bgcolor="#FFFFCC" width="161"> <font size="2"> <p><a href="#TechTalk"><b>Tech Talk</b></a><br> <a href="#Cracking Open Berkley DB">Cracking Open Berkeley DB</a> </p> <p><a href="#Editorial"><b>Editorial</b></a><br> <a href="#Arrogance and Greed">Arrogant and Greed<br> </a><a href="#Change and Growth">Change and Growth</a> </p> <p><a href="#Agile methodologies"><b>Agile methodologies</b></a></p><p><a href="#Book Review"><b>Book Review</b></a></p><p><a href="#Trends"><b>Trends</b></a><br> <a href="#New person, same job market">New person, same job market</a></font> </p> <p>  </td> <td height="277" bgcolor="#FFCC99" width="621"> <b><i><u><font FACE="Arial" COLOR="#fce016"> <h1><a name="TechTalk"><font color="#000000">Tech Talk</font></a></h1> </font></u> <h2><font face="Arial" color="#000000"><a name="Cracking Open Berkley DB">Cracking Open Berkeley DB</a></font></h2> </i></b> <p><i>By <a href="Author-JesusMonroy.html">Jesus Monroy, Jr.</a> <a href="mailto:(jessemonroy@email.com">(jessemonroy@email.com</a> )</i></p> <p>Berkeley DB (Data Base) is currently v2.x.x. There is a perception that it can only has limited use. It's best use is as persistent storage. And because it is a test library one can be reasonably assured of a quality. In short, if you'd like to "roll your own", Berkeley DB is excellent choice.</p> <p>In this article, we will explore the advantages and disadvantages, the services available, possible uses and some real applications.</p> <b> <h3>Advantages</h3> </b> <ul> <li>Library/Not a language</li> <li>Architecture-independent database</li> <li>Persistent and Volatile storage</li> <li>Native OS file storage</li> <li>Not a daemon/server</li> <li>A systems admin not required</li> <li>Learning curve is minimal</li> <li>Programming Interface is generic and easy</li> <li>Multiple APIs (C, C++, Perl, Tcl, Java)</li> <li>Thread-safe Library</li> <li>Large Key/Values pairs (up to 2^32 bytes)</li> <li>It's Open Source</li> </ul> <b> <h3>Disadvantages</h3> </b> <ul> <li>Does not deal with relational databases easily</li> <li>Complex data means complex program and storage issues</li> <li>Not network aware</li> <li>No network access</li> <li>No SQL interface</li> </ul> <b> <h3>It's More Than Just a Hash Table.</h3> </b> <p>From it's historic past DB, formerly DBM (and NDBM), was based on a key/data pair concept. The early releases also included the B+tree access method. Along with this, Fixed- and Variable-length Records were supported.</p> <p>Today, in addition to the aforementioned access methods, Berkeley DB supports a number of commercial improvements. Among them is shared memory-mapped caching and files. Transactions that use ACIDity (Atomic, Consistent, Isolatable, Durable) properties. Other features include:</p> <ul> <li>Write-ahead logging</li> <li>Crash Recovery</li> <li>Check pointing service (synchronization of logs against disk-writes)</li> <li>Two-phase locking (reduces deadlocks)</li> </ul> <p>In short, Sleepy Cat has added quit a few features that take Berkeley DB beyond the realm of “student work’.</p> <h3><b>Generic Interfaces</h3> </b> <p>Like many traditional programming interfaces Berkeley DB provides the minimal, which include:</p> <ul> <li>open() open the database in a prescribed fashion</li> <li>close() closes the database</li> <li>put() puts a record in database</li> <li>get() gets record from database</li> <li>del() deletes a record from the database</li> </ul> <p>In addition, there is:</p> <ul> <li>fd() returns a file handle for use with fcntl(2) and flock(2)</li> <li>sync() flushes any pending data to persistent storage</li> <li>seq() returns the database in a sequential manner, usually sorted</li> </ul> <blockquote> <b><font SIZE="2"> </blockquote> </font> <h3>Extendable Functions</h3> </b> <p>Except for open(), all the aforementioned routines are accessible via pointers in a structure returned by open(). In addition, since certain functions will never be "correct" for every situation, Berkeley DB has hooks for certain key routines. The include:</p> <ul> <li>hash() in hash, places the data in a hash of your creation</li> <li>compare() in btree, compares two (2) data items</li> <li>prefix() in btree, prefixes based on two (2) data items</li> </ul> <blockquote> <b><font SIZE="2"> </blockquote> </font> <h3>Granular Storage Control</h3> </b> <p>When using at open(), one can define the functional properties of put() based on "cursor" locations. One can:</p> <ul> <li>Replace the key/data pair referenced</li> <li>Append to the data immediately after the cursor</li> <li>Insert to the data immediately before the cursor</li> <li>Enter only if the key does not previously exist</li> <li>Store at the key/data pair, setting or initialize it</li> </ul> <b> <h3>Fast Access</h3> </b> <p>Berkeley DB obtains fast access by doing "block" reads and writes. As the term infers, DB reads and writes to the file system in blocks; handling all the parsing in memory. This includes records written with new lines ('\n') as record delimiters.</p> <p>You may set the parameters manually for run-time optimization by changing the following parameters:</p> <blockquote> <p><font FACE="Courier New">btree -> psize // pagesize<br> hash -> bsize // blocksize<br> recno -> psize // pagesize</font></p> <b><font SIZE="2"> </blockquote> </font> <h3>Possible Uses</h3> </b> <p>In the Berkeley DB man pages the term “store back” is used. In DB parlance this can be any storage device. In BSD UNIX systems we use the hard disk as a “store back”. Most other OSes do also.</p> <p>Some of the possible uses for Berkeley BD are:</p> <ul> <li>Persistent State Machine</li> <li>Application Configuration</li> <li>Transient Record Control</li> </ul> <p>In next section we'll actually write some code to do this.</p> <b><font SIZE="2"> <p> </p> </font> <h3>Some Applications</h3> </b> <p>Here are some of the applications that Berkley DB have been used to implement:</p> <ul> <li>Persistent State Machine.</li> <li>Application Configuration.</li> <li>Transient Record Control.</li> </ul> <i> <p>[Editor’s Note: This article was particularly relevant to me. I am working with a firm that uses Sleepy Cat (Berkley DB under commercial license for support and enhancements). It is used as a fast distributed backing store that front-ends an Oracle database backend.</p> </i> <b> <h1><font color="#000000"><a name="Editorial">Editorial</a></font></h1> </b> <i> <p>By <a href="Author-RegCharney.html">Reg. Charney</a></p> </i> <b><i> <p><font face="Arial"><a name="Arrogance and Greed">Arrogance and Greed</a></font></p> </i></b> <p>By now, you must have heard about the arrest and detention of Dmitri Sklyarov, the Russian programmer who wrote a decoder for Adobe’s eBook file format. He was arrested after giving a presentation at the Las Vegas Hacker’s conference. Dmitri was accused of contravening the Digital Millennium Copyright Act (DMCA) which says that it is illegal to digitally reproduce copyrighted material for any reason or to write software that can be used for that purpose. Dmitri wrote software for a Russian company that allowed Adobe’s eBook file format to be read on devices other than those originally supported. Based on Adobe’s complaint, the FBI arrested Dmitri and held him incommunicado for days without access to the Russian consul or to a lawyer. Is this right or are the Feds trying to intimidate us all based on their power and arrogance? Speaking of arrogance, the DMCA is a bad law, but also arrogant law. What right do the companies who paid for the DMCA have to impose U.S. laws on foreign firms, other than arrogance and greed? What Dmitri did in Russia was legal. What if you wrote a program here and went overseas for a vacation only to find yourself arrested because writing such a program which was illegal there? Are you to be a prisoner in the U.S. and in a given state, least one of your legitimate activities is illegal outside your immediate jurisdiction? The DMCA makes us all into potential criminals subject to unrestricted arrest and seizure. Look at Dmitri—you could be next.</p> <p><font FACE="Arial"><b><i><a name="Change and Growth">Change and Growth</a></i></b></font></p> <p>As you may have noticed, there are a number of new contributors to this issue of ACCent. It is with tremendous pleasure and pride that I can welcome Ali Çehreli, who is now doing the Trends page. Also, Jesse Monroy, Jr. has contributed immensely to this issue. He has written the main article and also done the book review. At this rate, he will soon replace me ;-).</p> <b> <h1><font color="#000000"><a name="Agile methodologies">Agile methodologies</a></font></h1> </b><i><font SIZE="2"> <p>By <a href="Author-AllanKelly.html">Allan Kelly</a></p> </font></i> <p>Most people who have studied software development will have learned about methodologies like Jackson JSD, Yourdon and SSADM., but somehow these methodologies belong to alternative world of software development, they are slow and lumbering, with formal hierarchies and lots of useless documentation. While many of us have experience parts of a methodology few of us have been unlucky enough to experience an entire project run on formal basis.</p> <p>While some methodology didn’t survive the coming of PCs and object oriented programming they have not gone away altogether. A few years ago Kent Beck came along with Extreme Programming (XP). Beck seemed to say: “rather than design a methodology and force people to work to it, lets take what works and structure it.” Observant readers of ACCU literature will know that I’m less than impressed by XP, but it does contain a lot of good ideas.</p> <p>Well, it turns out that XP is not alone, around the same time other people had similar ideas. They have adopted the title “Agile” and formed an alliance of like minded new methodologists, you can read their manifesto at <a href="http://www.agilealliance.org">www.agilealliance.org</a>  .</p> <p>The manifesto can seem a little abstract but fortunately Martin Fowler (author of several books on UML, XP and patterns) has done a good job of putting the whole thing in context in two papers, “The New methodologies” and “The Agile Manifesto: where it came from” both are available on his web site at <a href="http://www.martinfowler.com">www.martinfowler.com</a> .</p> <p>There are several agile methodologies to choose from but all have similar underpinnings: low ceremony, “lightweight”, short iterations, open to change and, perhaps most importantly: people centric. The traditional methodologies tended to treat developers as plug-compatible programming units, the agile methodologies put the people at the centre of the process. This dirty little secret of software development has long been know by us, the practitioners: all programmers are not created equal; however it always seemed that MBAs don’t teach this fact.</p> <p>Although not part of the Alliance Jim Coplien has conducted research in similar areas and produced a set of organisational patterns. More details at his web site <a href="http://www.bell-labs.com/user/cope">www.bell-labs.com/user/cope</a> .</p> <p>These methodologies and patterns are all in their early stages yet and subject to change. Most actually encourage practitioners to refine the process as they use it: this is something I have long believed, all methodologies are models, each must be tailored to your environment, your business model, your developers and your requirements. Long term, the acceptance of the process feedback loop may be even more dramatic than accepting that good people are important.</p> <b> <h1><font color="#000000" face="Arial"><a name="Book Review">Book Review</a></font><i> </h1> <p>By <a href="Author-JesusMonroy.html">Jesus Monroy, Jr.</a> <a href="mailto:(jessemonroy@email.com">(jessemonroy@email.com</a> )</p> <p><font FACE="Arial">Programming Embedded Systems in C and C++, 1st Ed. </font>by Michael Barr. Publisher: O'Reilly & Associates. ISBN: 1-56592-354-5</p> </i></b> <p>The book, in the preface, claims to be written for two types of readers: beginners - like college graduates; and embedded systems programmers.</p> <p>This is a good book for someone that knows something about programming C or C++, but does not know embedded systems.</p> <p>The coverage in the book was decent. However, embedded systems is a large topic and therefore a difficult one to cover. The author's coverage is light, at best. The author might have done better, if a different approach was taken, or even a different title was given to the book.</p> <p>There were four disincentives to checking code in this book:</p> <ol> <li>The code was extremely simplistic.</li> <li>The code compiled on a compiler I did not have (Borland).</li> <li>The example code was supposed to run on an embedded system I did not own.</li> <li>The author made no provision to provide the code in electronic format.</li> </ol> <p>The book does not rate well. I would mark it next to useless. I found several items particularly annoying.</p> <p>At the writing of this book, Linux was not available on many embedded systems. Choosing the Arcom Target188EB was probably as good as any. However, it is PC-104, making it unduly expensive - $275 retail.</p> <p>In one section of the book the author addresses the reader in a female tone. This is particularly annoying. Here is a clip:</p> <blockquote> <p>“After reading the first five chapters, she will be able to put her programing skills …”</p> </blockquote> <p>I’m for equal rights, but this is a particularly annoying moniker. I've encountered this type of writing before. It is either bad humor, bad writing, or just meant to annoy.</p> <p>This books fills a vacuum, but not in a very good fashion. The 140+ pages might be better spent with on-line examples or having the finally product do more than “Hello World!”.</p> <p>As a final note to the author, this is a good start. You writing style is good. The area intended for this book has a vacuum. You can fill it. I would suggest writing the next book with less expensive components - maybe even an obsolete system, like an 80286 PC, or a MAC SE. Or maybe a book with some generic *useful* libraries.</p> <b><i> <h1><font color="#000000"><a name="Trends">Trends</a></font></h1> </i></b> <i> <p>By <a href="Author-AliCehreli.html">Ali Cehreli</a></p> <b> <p><font face="Arial"><a name="New person, same job market">New person, same job market</a></font></p> </b></i> <p>Now it is my turn to be challenged to find interesting trends to write about in the technology jobs market. As Reg. has been repeating for many months now, the job market has been shrinking steadily. The only interesting thing has been the change of this change. This month is not different: Figure 1 shows how the drop in the job market has changed last month. The drop is actually dropping. Hopefully this will be more than just a hiccup like the one in March 2001, and the market will finally start to grow.</p> <p align="center"><img border="0" src="images/2001-08-Fig01.gif" width="468" height="274"></p> <p>I also want to look at some raw figures that display the relative differences in language demand, along with the dramatic change in the number of language related jobs that took place in the last two years.</p> <p align="center"><img border="0" src="images/2001-08-Fig02.gif" width="702" height="427"></p> <p>The interesting thing in this figure is how C/C++ and Java remain constant in September-October 2000 timeframe while the others continue to drop to a local dip only to recover temporarily. For C/C++ and Java, a bigger drop comes at around January 2001.</p> <p>Being a software engineer in the Silicon Valley, I’ve been witnessing the drop in the job markets as the now-very-usual news of layoffs here and there. Amazingly though the valley is still very fertile with the numerous startup companies still looking for strong engineering talents to beat their competition and come out on top. </td> </tr> </table> </body>