2002-06

<body stylesrc="index.html" background="images/Bkgrnd02T2.gif"> <table border="0" cellpadding="6" cellspacing="0" width="810"> <tr> <td align="right" valign="top" height="1" width="170"> <p align="center"><img border="0" src="images/Accent03.gif" align="left" width="142" height="37"> </p> <p align="left"><b><font size="2">June 2002 - Vol. 3 No. 6</font></b></p> </td> <td height="1" width="612"> <p align="center">  <a href="index.html" target="_top">[Home]</a> <a href="Newsletters.html" target="_top">[Up]</a> <a href="2002-07.html" target="_top">[Next]</a> <a href="2002-05.html" target="_top">[Previous]</a></td> </tr> <tr> <td valign="top" height="529" bgcolor="#FFFFCC" width="170"> <font size="2"> <p><a href="#TechTalk"><b>Tech Talk</b></a><br> <a href="#Append Utility">Append Utility</a> </p> <p><a href="#Editorial"><b>Editorial</b></a><br> <a href="#Hammers, Nails and Technology">Hammers, Nails and Technology<br> Preserving Your Rights with Art</a><a href="#Call for articles"><br> Call for Articles</a> </p> <p><a href="#Book Reviews"><b>Book Reviews</b></a><br> <a href="#Standard C++ IOStreams and Locales">Standard C++ IOStreams and Locales</a> </p> <p><b><a href="#Humor">Humor</a></b><a href="#eMail Signatures"><br> eMail Signatures</a></p> <p><a href="#Trends"><b>Trends</b></a><br> <a href="#Behind the rest of the country">Behind the rest of the country</a></font></p> </td> <td height="277" bgcolor="#FFCC99" width="612"> <b> <h1><a name="TechTalk"><font color="#000000">Tech Talk</font></a></h1> <h2><font face="Arial"><a name="Append Utility"><font color="#000000">Append Utility</font></a></font></h2> </b> <i> <p>By <a href="Author-RegCharney.html">Reg Charney</a></p> </i><b> <h3><font FACE="Arial">Requirements</font></h3> </b> <p>I recently needed to append a series of files to a 50MB output file. While it is possible to combine these files using some editors. or using the cat command, or using redirection, but most of these tools have a significant drawback. They require 50MB+ or more of memory or temporary disk space. This was a problem for me because I did not have the extra 50MB to spare.</p> <b> <h3><font FACE="Arial">Feature Set</font></h3> </b> <p>I modeled this program on the Unix <font face="Courier New" color="#008000">mv</font> and <font face="Courier New" color="#008000">cp</font> commands. A list of input files are followed by a target file. The input files are appended to the target file. If the target file does not exist, it is created. If an input file does not exit or can not be read, it is ignored. I also decided to write this program in C++ using object-oriented techniques.</p> <b> <h3><font FACE="Arial">Problems to Overcome</font></h3> </b> <p>There are a few “ugly” things about most implementations (you may have others ;-). One of them is the need to expose the buffer and its size as shown in the main loop pseudo-code below:</p> <div align="center"> <center> <table border="1" width="44%" bgcolor="#FFFFFF"> <tr> <td width="100%"><font FACE="Courier New">open the output file<br> for each input file<br> {<br>   open input file<br>   iSZ = size of input file<br>   allocate char[iSZ] buffer<br>   read file into buffer<br>   write output from buffer<br>   close input file<br>   free buffer<br> }</font></td> </tr> </table> </center> </div> <p>This doesn’t appeal since it exposes implementation details at too high a level. A second problem is less obvious. What happens if one of the input files is also the output file? Do we get an infinitely large file? The third problem concerns the size of the files. The fragment shown shows a simple implementation that assumes it is possible to read any sized input file into memory. Of course, this is naïve. Ideally, this should not be a concern at the highest level of the program.</p> <div align="center"> <center> <table border="1" width="85%" bgcolor="#FFFFFF"> <tr> <td width="100%"><font FACE="Courier New">#include <stdio.h><br> #include <fstream.h><br> #include <memory></font> <p><font FACE="Courier New">const int MAXBUF = 1024;</font></p> <p><font FACE="Courier New">class FBuf {<br>   protected:<br>     static char *bp;<br>     static int bSZ;<br> };<br> char* FBuf::bp  = 0;<br> int   FBuf::bSZ = 0;</font></p> <p><font FACE="Courier New">class OFile : public FBuf {<br>     ofstream oF;<br>     char* oFN;<br>   public:<br>     OFile(char* oFN_) : oFN(oFN_)<br>     {<br>       oF.open(oFN, ios::app|ios::binary,<br>               filebuf::sh_none);<br>       if (!oF)<br>         throw oFN;<br>     }<br>     ~OFile()<br>     {<br>       oF.close();<br>     }<br>     void write()<br>     {<br>       if (oF.is_open())<br>       {<br>         oF.write(bp, bSZ);<br>         delete [] bp;<br>         bp = 0;<br>         if (!oF)<br>           throw oFN;<br>       }<br>     }<br> };</font></p> <p><font face="Courier New">class IFile : public FBuf {<br>     char* iFN;<br>     ifstream iF;<br>     int fSZ;<br>     int iP;<br>   public:<br>     IFile(char* iFName) <br>       : iFN(iFName), fSZ(0), iP(0)<br>     {<br>       iF.open(iFN, ios::in|ios::binary);<br>       if (!iF)<br>         throw iFN;<br>       iF.seekg((streampos)0, ios::end);<br>       fSZ = iF.tellg();<br>       iF.seekg((streampos)0, ios::beg);<br>       if (!iF || fSZ == 0)<br>         throw iFN;<br>       bSZ = (fSZ < MAXBUF) ?<br>       fSZ : MAXBUF;<br>     }<br>     ~IFile()<br>     {<br>       delete [] bp;<br>       bp = 0;<br>       iP = 0;<br>       iF.close();<br>     }<br>     operator bool()<br>     {<br>       return (iP < fSZ);<br>     }<br>     char* read()<br>     {<br>       try<br>       {<br>         int oldP = iP;<br>         int newP = iP + bSZ;<br>         iP = (newP < fSZ) ? newP : fSZ;<br>         bSZ = iP - oldP;<br>         if (bSZ)<br>         {<br>           bp = new char[bSZ];<br>           if (!bp)<br>             throw iFN;<br>           iF.read(bp, bSZ);<br>         }<br>       }<br>       catch (char *s)<br>       {<br>         printf("Error processing input file "<br>                "%s\n", s);<br>         throw;<br>       }<br>       catch (...)<br>       {<br>         ; // ignore all else<br>       }<br>       return bp;<br>     }<br>     int sz()<br>     {<br>       return iF.gcount();<br>     }<br> };</font></p> <p><font FACE="Courier New">int main(int argc, char* argv[])<br> {<br>   //* check cmd line args<br>   if (argc < 3)<br>   {<br>     printf("\n\nUsage: append in-file1 "<br>            "[ , in-file2 ]..."<br>            "output-file\n\n");<br>     return 1;<br>   }</font></p> <p><font FACE="Courier New">  OFile oF(argv[argc-1]);</font></p> <p><font face="Courier New">  // loop over each argument, except the last arg<br>   for (int i=1;i<argc-1;i++)<br>   {<br>     try <br>     {<br>       IFile iF(argv[i]);<br>       while (iF)<br>       {<br>         iF.read();<br>         oF.write();<br>       }<br>     }<br>     catch (char *s)<br>     {<br>       printf("Error in file '%s'\n", s);<br>     }<br>     catch (...) // catch all<br>     {<br>       continue;<br>     }<br>   }</font></p> <p><font face="Courier New">  return 0;<br> }</font></td> </tr> </table> </center> </div> <b> <h3><font FACE="Arial">Problem Solutions</font></h3> </b> <p>We can use the fact that this application must read and write the input files in order. That is, only one input file is opened at a given time. Thus, it is valid to use a static buffer pointer and size for all files. Also, since both the current input file and the output file can use the same buffer, we have defined <font face="Courier New" color="#008000">OFile</font> and <font face="Courier New" color="#008000">IFile</font> as descending from <font face="Courier New" color="#008000">FBuf</font>. <font face="Courier New" color="#008000">FBuf</font> contains two protected static data members. Thus, every instance of <font face="Courier New" color="#008000">IFile</font> and the instance of <font face="Courier New" color="#008000">OFile</font> share the same buffer pointer and the size of the current input file. This eliminates the need to expose the implementation in the main function.</p> <p>The second potential problem occurs if one of the input files is also the output file. While the current implementation may read in the whole file before trying to write the output, we could have a problem is the input file is read in pieces because of size constraints. The normal way of preventing the use of input and output files overlapping is to compare the full UNC of each input file against the UNC of the output file. However, there is simpler solution in this case. The output file is opened for exclusive use. Thus, any attempt to open the output file for input will fail. In this case, this is fine.</p> <p>Lastly, we may encounter files that are too large to fit in memory. Since the details with respect to file size are hidden in <font face="Courier New" color="#008000">FBuf</font>, <font face="Courier New" color="#008000">IFile</font>, and <font face="Courier New" color="#008000">OFile</font>, the main program can basically ignore these details. The concession to the possibility of large files is the inner loop that will read and write as long as there is input in the input file. The constant <font face="Courier New" color="#008000">MAXBUF</font> can be set to any value. If the size of the file is less than <font face="Courier New" color="#008000">MAXBUF</font>, the whole file will be read in just once.</p> <b> <h1><font color="#000000"><a name="Editorial">Editorial</a></font></h1> </b> <i> <p>By <a href="Author-RegCharney.html">Reg. Charney</a></p> </i><b> <h2><font FACE="Arial"><a name="Hammers, Nails and Technology">Hammers, Nails and Technology</a></font></h2> </b> <p>To a hammer everything looks like a nail. As technologists, we believe any problem can be solved using technology. Unhappily, this is not true. We need to be aware of our environment. Thus, Dan Gillmor's presentation on Intellectual Property rights was so important. As he outlined, not only will you need to solve problems, but you may now need to solve them in unique and/or non-optimum ways if IP issues keep locking up innovative or trivial ways of doing things. Not only will your manager control your code production, so will the company lawyers. They don’t know anything about programming and couldn't care less. Also, think of all the paperwork you will need to do to protect your code as part of the company's IP portfolio. But you don’t need to worry — all this is not technical! You don’t need to get involved, join the Electronic Freedom Foundation (<font FACE="Arial"><a href="http://www.eff.org">www.eff.org</a></font>) or join groups like DVD-DISCUSS (<font FACE="Arial"><a href="http://eon.law.harvard.edu/archive/dvd-discuss/">http://eon.law.harvard.edu/archive/dvd-discuss/</a></font>) to see what you can do to protect yourself and those you love.</p> <b> <h2><font FACE="Arial"><a name="Preserving Your Rights with Art">Preserving Your Rights with Art</a></font></h2> </b> <p>Unless you have been living in a cave for the last couple of years, you know that your rights to “fair use” of CDs and DVDs that you have purchased for your personal use in your own home have been restricted by a court that believes that antiquated revenue models from media corporations take precedence over your “fair use” rights. To the rescue comes art! See the sites listed in <font FACE="Arial"><a href="http://web.lemuria.org/DeArt/Sep/">http://web.lemuria.org/DeArt/Sep/</a></font> for art examples. Art also comes to the rescue to allow you to play DVDs on any player, including your computer. See how at <font FACE="Arial"><a href="http://www.theregister.co.uk/content/54/25274.html">http://www.theregister.co.uk/content/54/25274.html</a></font>.</p> <b> <h2><font FACE="Arial"><a name="Call for articles">Call for Articles</a></font></h2> </b> <p>We are always looking for articles. If you are interested in writing about your scripts or applications in other languages, we would be happy to consider publish them,</p> <b> <h1><font color="#000000"><a name="Book Reviews">Book Reviews</a></font></h1> <i> <p><a name="Standard C++ IOStreams and Locales">Standard C++ IOStreams and Locales </a></i> </b> by Angelika Langer and Klaus Kreft, Addison-Wesley, ISBN 2000, 0-201-18395-1</p> <p>This book is aimed at advanced C++ programmers. Half of the book is an extremely detailed examination and explanation of IOStreams, internationalization and locales. The other half is a reference guide.</p> <p>The authors state their goal as to “focus on the underlying concepts and the more advanced programming techniques that IOStreams and locales support.” They succeed at this, covering these topics thoroughly, providing both a good overview and a wealth of low level details. I was glad to see eight pages devoted to “The Stream State”, an important topic for IOStreams users, but one that is often omitted. They provide many useful tips not seen elsewhere, and point out traps and pitfalls.</p> <p>I was only able to find one minor error which was not already listed in the errata on the author's web site. For such a complex and detailed book, this is quite impressive.</p> <p>The index was shorter than I expected in a book of this scope, but in practice I was able to find what I needed in the index, so it seems sufficient.</p> <p>I don't know of any other book or reference material that covers IOStreams in nearly this complete and thorough a manner.</p> <p>However, locales are also covered in Appendix D of Stroustrup’s “The C++ Programming Language” (also available from Stroustrup’s web site), which is a 64 page tutorial description of locales. Advanced usage of locales is sufficiently complex that anyone attempting it should refer to both this book and to Stroustrup's appendix.</p> <p>The writing is precise, yet readable. This book is essential for all interested in advanced usage of IOStreams or of locales.</p> <p><i>— Wayne Vucenic</i></p> <i> <b> <h1><font color="#000000"><a name="Humor">Humor</a></font></h1> </b> </i> <b> <h2><font FACE="Arial"><a name="eMail Signatures">eMail Signatures</a></font></h2> </b> <p>A colleague of mine once programmed on-board satellite controllers. On the project was a mechanical engineer whose job is was to track the weight of the payload. Every week he would ask “how much does the software weigh.” And every week my friend would say “Nothing, go away.” Then one week the payload petty officer arrived with a large box of Hollerith cards, indignantly proclaiming “These cards weigh over eight pounds!” My friend said “You don't understand. The software isn't the cards, it's the holes.” The still-mystified mechanical engineer left, never to ask about the weight of software again.</p> <p><i>— </i>from Greg Colvin’s email signature</p> <p>May I have the serenity to accept the things I cannot change, the strength to change the things I cannot accept, and the cunning to hide the bodies of those who got in my way.</p> <p><i>— </i>D. C. Sessions’ email signature</p> <i> <p>[Ed. If you come across signatures that you think are humorous, send me (</i><a href="mailto://editor@accu-usa.org">editor@accu-usa.org</a>) <i>a line. Other readers may enjoy them too.]</i></p> <i> <b> <h1><font color="#000000"><a name="Trends">Trends</a></font></h1> </b> <p>By <a href="Author-AliCehreli.html">Ali Çehreli</a></p> </i> <b> <h2><font FACE="Arial"><a name="Behind the rest of the country">Behind the rest of the country</a></font></h2> </b> <p>The total number of available jobs nationwide has increased, but the jobs in the valley, as well as the software jobs both nationwide and in the valley have dropped (Figure 1). Even though the number of jobs concerning almost all programming language has dropped in the valley, C++ came out to be the most in demand because of having a lesser drop (Figure 2).</p> <p align="center"><img border="0" src="images/2002-06-Fig01.gif" width="466" height="291"></p> <p align="center"><img border="0" src="images/2002-06-Fig02.gif" width="462" height="351"></p> <p>Among our data points, SmallTalk and Lisp/Clos have both increased their figures from no jobs to only one job this month. Delphi has been another language with good performance: up to 8 jobs from the last month's 4. Obviously these figures are too small to be considered phenomenal.</p> </td> </tr> </table> </body>