HISTORY.TXT (for DosLynx v0.31b, May 2006) by Fred C. Macall Introduction This is for my tenth release of DosLynx. In this document, I'll try to take up where I left off in the last history.txt and not rehash the history of the previous versions. However, there is a cumulative list of unresolved bugs, issues, and To Do(s) at the end. If you want to know everything else there is to know about my previous releases of DosLynx, you can get most of that history from: http://users.ohiohills.com/fmacall/dlx2xdoc.zip . This archive collects the info.htm and history.txt documents for all of my DosLynx v0.2xb releases. For the rest, the documents for DosLynx v0.30b are available at: http://users.ohiohills.com/fmacall/dlx30/info.htm and http://users.ohiohills.com/fmacall/dlx30/history.txt In February, I was able to add a Windows 2000 Pro, SP3, installation to my Pentium based home networked PC, Digerydo. Digerydo runs at 60 Bogo MIPS and has 64 MB of RAM. This new installation has provided my first Windows 2000 DOS window experience with DosLynx. I am pleased to note that no unpleasant surprise came from DosLynx, in this new experience. However, I have noted that a fresh approach may be required for providing graphics support, for better than VGA resolutions, in the Windows 2000 DOS window. This new installation has also enabled my first experience with SWSVPKT.SYS. That is described on my Web site's updated What To Do with a WinModem page: http://users.ohiohills.com/fmacall/winmodht.htm . Old Business Or, Protected Mode Version, this time. I started the work for DosLynx v0.31b by adding a declaration for extern unsigned long _protheaplen to GLOBALS.H. _protheaplen specifies the size desired for the DosLynx Protected Mode version's heap of DPMI provided XMS memory. As proposed in the last HISTORY.TXT, I also instanciated it, in GLOBALS.CPP, as: unsigned long _protheaplen = 0x00260000L . (That provides about 2.3 MB.) I was able to eliminate most of the DosLynx Protected Mode version's baby fat by adding an /A=0016 option to the TLINK command. The need for that was easy to figure out once I was able to find a manual with a good TLINK reference. I eliminated a few KB more of fat, from the Protected Mode version's .EXE, by #ifdef(ing) four kinds of routines in DPMISH.CPP and DPMISH.H. I am distinguishing routines used nowhere. Routines used only in debug version(s). Routines used only in DPMIRUN. And, routines used only in DOSLYNX. Remember that the DosLynx Protected Mode version's .EXE includes two links: DOSLYNX.EXE and, its "Real Mode Stub", DPMIRUN.EXE. The routines in the first two categories just mentioned now get left out of both of those links. The routines in the last two categories just mentioned now get left out of one link or the other. To accomplish this saving, I also had to use the TLIB utility to remove DPMISH.OBJ from PMAPP's CLIB.LIB. After that, DpmiApplication Dpmi needed to be instanciated somewhere. I added that to TDOSLYNX.CPP. The differential between the sizes, of the DosLynx Protected and Real Mode versions, has now been reduced to about what is expected to be its minimum. With the DosLynx Protected Mode version's size no longer an issue, I turned to analyzing and resolving its performance issues. That is, performance issues not seen in the DosLynx Real Mode version. In the end, I only found three kinds of these: 1. That long delay experienced, with some DPMI service(s), upon the first use of TCP/IP in a session. This delay lasted eighteen seconds, in a QEMM/QDPMI environment, on my 16 Bogo MIPS '486 based development PC, Sailboard! 2. Distributed delays attributable to poor malloc( ) and free( ) performance. These were particularly noticeable when large cached documents got free(d). 3. Unsatisfactory performance experienced in areas which haven't been reached in the DosLynx Real Mode version, due to its memory limitations. For example, when receiving ftp directory reports too large for the Real Mode version to memorize. Well, that first issue turned out not to be coming from any of the PCPKT.C routines that I had originally suspected. Instead, it proved to be coming from tcp_config( ), in PCCONFIG.C. There, the DOSLYNX.CFG file was being read, a character at a time, by means of a _read(f, & ch, 1) call. The comments in my DOSLYNX.CFG file had brought it to a size of about 28.6 KB. I confirmed the situation by doing a test with the 757 octet DOSLYNX.CFG file that remained after removing most of the commented file's comments. I concluded that the QEMM/QDPMI environment's overhead limits a Protected Mode caller's disk I/O calls (to DOS) to a rate of only about 1500 per second, on Sailboard! The DosLynx README.HTM already advises users of older (read: slower) PCs to remove the comments from their DOSLYNX.CFG file. But, did I want to have to recommend this to users of shiny and fast newer PCs, too? Well, I guess not. I decided to add a bufread( ) routine, for buffering tcp_config( )'s calls to _read( ), to the Protected Mode version of PCCONFIG.C. The original _read( ) call has been replaced with a readfile( ) call that gets #define(d) either to _read( ), in the Real Mode version, or to bufread( ), in the Protected Mode version. This solves the problem at hand without adding to the size of the DosLynx Real Mode version. I studied PMAPP's MALLOC.CPP for the purpose of finding way(s) to resolve that second issue and concluded that excessive complexity was the main problem, there. I decided to replace MALLOC.CPP, in the DOSLYNX.EXE Protected Mode version make, with a new module named MYALLOC.CPP. MYALLOC.CPP is strictly focused on managing a heap of XMS memory obtained through the DPMI service. This module's struct header, malloc( ), and free( ) loosely follow the example given in section 8.7 of: Brian W. Kernighan/Dennis M. Ritchie, The C Programming Language (first edition). Here is a copy of my struct header, to be compared with the example's: /* Header for all free and allocated memory blocks. */ struct header { int magic ; /* Eyeball for heap validity check.*/ unsigned int sizepars ; /* Size (paragraphs). */ /* If 0, block won't be allocated */ /* nor combined with a free block. */ /* Where the user's data starts, in an allocated block: */ struct header * pnextfre ; /* If free, A(next free block). */ struct header * pprevfre ; /* If free, A(previous free block).*/ long int spare ; /* Pad making sizeof(struct header)*/ /* equal to parasize. */ } ; It may be seen that my copy is quite loose! Also of note, in my arrangement, is that free memory blocks are doubly linked, while allocated memory blocks go unlinked. In limiting the memory management overhead in allocated memory blocks to only four octets, my arrangement shares a part of the Borland C/C++ v3.1 runtime's approach. The double linkage, for free memory blocks, provides for malloc( ) and farheapcheck( ) implementations that have an easy way to watch for an unexpected short loop in the forward linkage that they walk. My malloc( ) and free( ) differ from the example's alloc( ) and free( ) in two or three more notable ways. The first difference(s) stem from the fact that, in Protected Mode, there are a relatively limited number of Selector numbers available for describing the allocated memory blocks. As a practical matter, the Selector part of the pointer to each allocated memory block must match a Selector describing a block or Segment of the XMS memory obtained from the DPMI service. And, the Offset part of the pointer to each allocated memory block, plus the block's length, must not overrun that containing block or Segment of the XMS memory obtained from the DPMI service. In other words, the memory blocks being managed and allocated can't span multiple blocks or Segments of the XMS memory obtained from the DPMI service. In a 16 bit program, the blocks or Segments of the XMS memory obtained from the DPMI service are limited to a length of 64 KB. My arrangement for meeting this Segmented Memory Requirement starts in my replacement for the example's morecore( ). It divides each block or Segment of the XMS memory obtained from the DPMI service into two free memory blocks. The first is an empty or header only block located at the beginning of the obtained memory Segment. The second is a block containing all of the rest of the obtained Segment's memory. As empty memory blocks can't participate in allocation, nor be combined with block(s) being freed, their presence takes care of meeting the requirements spelled out in the preceding paragraph. By the way, my replacement for the example's morecore( ) looks more like the PMAPP MALLOC.CPP's CreateHeap( ). Because, it attempts to obtain all of the _protheaplen specified memory in a single run. The Segmented Memory Requirement leads to a speed-up in my free( ). The Selector part of the pointer to a memory block being free(d) must locate an empty free memory block, by itself. This is the empty free memory block at the beginning of the Segment that contains the block being free(d). free( )'s search of the list of free memory blocks starts with this empty free memory block. Doing that saves a lot of time in finding where to insert the memory block being free(d) (back) into the free list. The second or third notable difference between the example and my implementation arises from the former's use of the allocp variable. In the example, allocp locates a point in the list of free memory blocks where alloc( ) starts its search for a block big enough to meet the given request. Kernighan and Ritchie state that "this strategy helps keep the [free] list homogeneous." Well, I tried and quickly discarded that strategy. Because, together with the Segmented Memory Requirement and my preprocurement of the _protheaplen defined amount of memory, it also seemed to lead to a serious limitation. It seemed to result in quickly fragmenting all of the blocks or Segments of the XMS memory obtained from the DPMI service. The length of the longest free memory block available quickly diminished as a result! Even while much of the heap remained to be allocated. Instead, my malloc( )'s search always starts from static struct header base. This may lead to somewhat longer or more variable searches. But, it seems to keep most of the blocks or Segments of the XMS memory obtained from the DPMI service intact for about as long as possible. Very long free memory blocks seem to remain available until the heap is nearly full. I have been pleasantly surprised by MYALLOC.CPP's performance. It seems to have restored nearly all of the DosLynx Real Mode version's pep to the DosLynx Protected Mode version! Testing, done since resolving the memory management performance issue, has identified a few areas of unsatisfactory performance that previously hadn't been recognized. These all involve large processing tasks that have been beyond the DosLynx Real Mode version's memory capacity. These issues are the third kind of DosLynx Protected Mode version performance issues given in the list way back up near the beginning of this section. The large ftp directory report performance issue, given as an example above, is typical. I think I may have seen some signs of this issue in the DosLynx Real Mode version. But, I didn't really recognize it until I did some testing with really big ftp directory reports that the Real Mode version couldn't handle. The symptoms of this issue were an apparent "freeze" and a data channel timeout after receiving a long ftp directory report. This problem was found to lie in read_directory( ), in HTFTP.C. The received ftp directory report gets memorized in an HTBTree object as it is received. But, at the end of reception, that HTBTree had to be sent through the HTML parse before concluding NETCLOSE(data_soc) and response(NIL) calls got made. I have resolved this issue by moving those NETCLOSE(data_soc) and respose(NIL) calls up, to precede the HTBTree object's parsing, in read_directory( ). And, I've added an fcyield( ) call to the loop that unloads the HTBTree object into the parse process. That keeps the system from being unresponsive during the time needed for that parse. These changes may benefit the DosLynx Real Mode version, as well as the DosLynx Protected Mode version. The greatest benefit will come with a slow machine or when the DosLynx Real Mode version encounters the longest ftp directory reports that it can handle. fcyield( ) was introduced, under the heading: FCYIELD.CPP, in the history.txt for DosLynx v0.25b. Another area of unsatisfactory performance, when handling very large data object(s), was found in TURLView::setTArea( ), in TURLVI42.CPP. This routine filters the end-of-line sequences in a Form Textarea object's text data and updates the object's presentation, after it has been edited. 'setTArea( ) was introduced in the history.txt for DosLynx v0.27b. 'setTArea( ) copies the text of the Textarea object in process a line at a time and gives each copied line a standard end-of-line sequence. A new char[] allocation and StrAllocCat( ) call were being performed for each line of text copied. The DosLynx Protected Mode version can handle Textarea objects containing upto about 60 KB of text. Even the improved memory management arrangements couldn't keep this process from being slow when processing very large Textarea objects. A performance boost, though perhaps not as much as I would like, has been achieved by moving the new char[] allocation and StrAllocCat( ) call out of the loop over lines of text. A couple more Protected Mode violations have been experienced since the release of DosLynx v0.30b. One, that hasn't reoccurred, seemed to come from FRAGMENT.C. I can't quite explain that one. However, I did find three apparent off-by-one errors, in FRAGMENT.C. To correct these, I have changed data_length + 1 to simply data_length in two places. And, I have changed data_end to data_end + 1 in the assignment to ip->length. A Protected Mode violation, that could be reproduced at will, came from trying to use the Help|Mail Developer menu entry or command. (How did I miss seeing that before releasing DosLynx v0.30b?!) This was coming from the TILp_subject->setData( ) call to TInputLine::setData( ) that I revised for DosLynx v0.27b, in TDosLynx::mailDeveloper( ) in TDOSLY15.CPP. 'setData( ) uses a memcpy( ) call to fill its buffer, without watching for null(s) in the given string. (How would I know this if I were treating Turbo Vision strictly as a "black box"?) In the troublesome case, 'setData( ) was being given a string contained in a buffer on the Stack. That buffer's length is only a fraction of the length of the class TInputLine buffer that is being initialized. As a result, the memcpy( ) call was encountering Segment Overrun as it proceeded to copy data from beyond the end of the Stack. I fixed this problem by adding a strncpy( ) call to copy TDosLynx::mailDeveloper( )'s parameter string into its big buffer, cp_buffer, before calling 'setData( ). I also fixed the setup for an additional potentially troublesome 'setData( ) call, in 'mailDeveloper( ), the same way. I also fixed the setup for potentially troublesome calls, to TInputLine::setData( ), in HTALERT.CPP, TURLVI23.CPP, TURLVI28.CPP, and TURLVI30.CPP. In each of these modules, the "big buffer" is auto char ca_buffer[usi_TILURLSize], which is allocated on the Stack. In each case, I've added a strncpy( ) call to copy the parameter for 'setData( ) into ca_buffer[ ]. In HTPrompt( ), in HTALERT.CPP, ca_buffer[ ] was already declared in the containing function's outer block. In the other three modules, the fix necessitated moving ca_buffer[ ]'s declaration to the outer block of the containing function. Precautions Against Buffer Overrun in HTTP.C I am sure that, if you have ever looked at HTLoadHTTP( ) in HTTP.C, you must have noticed its elaborate arrangements for reading the first line of an HTTP header. But, did you notice that those elaborate arrangements afforded a "malicious Web site" at least two ways to make trouble for DosLynx? I admit that it took me a long time to recognize those issues. The trouble I finally noticed came from allowing the lengths of binary_buffer and text_buffer to double without limitation. The first problem could come once the quantity (buffer_length - length) became more than about 1024. From that point, the NETREAD( ) call being issued became capable of delivering an end-of-line sequence followed by more than 1024 octets of additional data. After such a delivery, HTLoadHTTP( )'s potential call to HTAA_shouldRetryWithAuth( ) would yield a buffer overrun, in HTAA_setupReader( ) in HTAAUTIL.C. In addition, a still malicious but less sophisticated Web site might arrange to send enough data, without an end-of-line sequence, to exhaust DosLynx's heap memory! To avoid that first vulnerability, I've revised the update for buffer_length to read: buffer_length = length + INIT_LINE_SIZE ; So, as long as INIT_LINE_SIZE is no more than 1024, a buffer overrun won't be possible in HTAA_setupReader( ). Avoiding that second vulnerability simply depends on choosing a reasonable maximum length for the HTTP header's first line. My fix extends HTLoadHTTP( )'s do { . . . } while ( . . . ) tests to read: while ((!eol) && (!end_of_file) && (length < 2048)) ; That will limit the lengths of binary_buffer and text_buffer to a maximum of about 3 KB. Tag and Style Stack Precautions in HTML.C HTML_start_element( ) and HTML_end_element( ), in HTML.C, maintain a Stack of tags and styles for the HTML document being parsed. HTML_put_character( ) and HTML_put_string( ), also in HTML.C, refer to this Stack's current element, as well. The Stack's size, 200 entries, might seem large enough to handle every possible HTML document. But, I have now seen at least one document that couldn't be handled! This document has a lot of missing end tags. That is, missing tags of the form: . While this document has an unexceptional total number of HTML tags, it may seem to have exceptionally deep tag nesting. HTML_start_element( ) guards against overflowing this Stack and produced the message: HTML: *** Maximum nesting of 200 exceeded! each time it omitted pushing another Stack entry onto an already full Stack. However, additions I made long ago, in SGML_free( ) and SGML_abort( ) in SGML.C, eventually replace a document's missing end tag(s). These additions were discussed under the heading: A Few Zits More, in the history.txt for DosLynx v0.22b. So, the troublesome document leads to HTML_end_element( ) being called, repeatedly, to pop Stack entries from what becomes an empty Stack. As there was no protection against that, in HTML_end_element( ), a crash resulted. #ifdef CAREFUL code, long present at the beginning of HTML_end_element( ), doesn't provide an appropriate response to the problem(s) identified here. However, it did provide some valuable insights about what needed to be done. My fix for HTML_end_element( ) does begin with a check for an empty Stack, of course. Finding that, it now omits its Stack pop. However, there is an extension for the case of a non-empty Stack, too. Instead of an automatic Stack pop, there is a new check to see if the tag in process matches the tag to be popped from the top of the Stack. Normally, there will be a match and the Stack pop will be done, as always before. However, there may not be a match after HTML_start_element( ) has withheld push(es) to avoid Stack overflow. In that case, a Stack pop isn't appropriate and gets omitted. At some point, during a recovery from suppressed Stack overflow(s), enough pops will have been omitted to bring the Stack back into accord with the document in process. This arrangement isn't as efficient as possible. But, it seems to go about as far as is reasonable, in trying to improve the presentation of what must be badly flawed documents. While working in HTML.C, I decided that having that *** Maximum nesting . . . exceeded message coming out for every suppressed Stack overflow wasn't very helpful. So, I've added a BOOL nestovrf to struct _HTStructured, in HTML.C. nestovrf gets zeroed, initially, by HTML_new( ), in HTML.C. Upon suppressing a Stack overflow, HTML_start_element( ) consults and then sets nestovrf. It only issues the message upon finding that nestovrf had been zero. HTML_end_element( ) zeros nestovrf (again), if it ever finds the Stack empty. These arrangements probably keep the message from appearing more than once for each troublesome document. Carriage Return Filtering for Presented Documents The document presentation processes in DosLynx assume that end-of-line will be signalled by means of an ASCII Line Feed (0x0A) character. In the DOS C/C++ environment, this character is also known as New Line or '\n'. This assumption is very strong in some of class TURLView's member functions. There, any Carriage Return (0x0D or '\r') character(s) present disrupt the presentation by blanking the remainder of any line(s) that contain them. As Carriage Return takes the role of New Line in some other computing environment(s), there are plenty of ways they can get into documents to be presented by DosLynx. A further complication is that DOS, outside of C/C++, uses a Carriage Return Line Feed sequence as its standard end-of-line signal. Upon invesitgating cases of apparent data loss in DosLynx document presentations, recently, I found Carriage Return character(s) to blame. They weren't being systematically filtered from HTML documents being received via TCP/IP based HTTP. Elsewhere in DosLynx, they were being systematically filtered by simple deletion. But, that wasn't always satisfactory, either. As when the Carriage Return(s) were coming from an environment using them to signal New Line. I decided to solve these problem(s), for HTML documents, by adding a more elaborate Carriage Return filter to the very beginning of PUBLIC void SGML_character(HTStream * context, char c), in SGML.C. My new filter depends on a BOOLEAN hadcr variable added to SGML.C's struct _HTStream. context->hadcr gets initialized to FALSE by SGML_new( ), also in SGML.C. My new filter reads as follows: /* Filter all CR character(s) given, as follows: /* Discard any CR(s) immediately followed by NL(s). /* Convert any other CR(s) to NL(s). */ if (context->hadcr) { context->hadcr = FALSE ; if (c == '\n') return ; } if (c == '\r') { context->hadcr = TRUE ; c = '\n' ; } This filter has proven to be a completely satisfactory solution for the problem(s) I was studying. After a month of experience with this filter, I made the following additional changes, in the interests of consistency: I revised HTLoadFile( ), in HTFILE.C, to open( ) local input files only with the O_BINARY option. This change also solves a data loss problem attributable to Ctrl-Z character(s) encountered in yEncoded data. Ctrl-Z caused an unintended end-of-file indication when read without O_BINARY treatment. The Carriage Return filtering that was being achieved by using non-binary or "cooked" file input will now be provided by the filter added to SGML_character( ). Or, in one of the following three other places: I revised HTNGetCh( ) (formerly HTGetChararcter( ) [sic]) and NetToText_put_character( ), both in HTFORMAT.C, to do their Carriage Return filtering the way the sequence displayed above does. HTNGetCh( ) had been deleting all Carriage Return characters. NetToText_put_character( ) had been deleting Carriage Return only when followed by Line Feed. It wasn't changing other Carriage Return character(s) to New Line character(s). I also extended HTPlain_put_character( ), in HTPLAIN.C, with a Carriage Return filter like the one displayed above. As in SGML.C, this involved the addition of BOOL had_cr to HTPLAIN.C's struct _HTStream. And, the addition of initialization, for me->had_cr, to HTPlainPresent( ) in HTPLAIN.C. Finally, HTPlain_write( ) and HTPlain_put_string( ), also in HTPLAIN.C, both had to be extended to do their work by means of call(s) to HTPlain_put_character( ). Paste File Push Buttons The DosLynx Send Mail dialog can accept about 4 KB of text. And, the Textarea Input Form control or dialog can accept up to about 60 KB of text, free heap memory permitting. Some means for reading text file(s) into these dialogs seems to be only a minimal amenity. I decided to provide this new facility via Paste File push buttons added to those two dialogs. TDosLynx::mailDeveloper( ), in TDOSLY15.CPP, and TURLView::edTArea( ), in TURLVI43.CPP, each needed to have code for a new push button added, of course. But, I tried to put most of the rest of the code for the new facility into a new function: void TDosLynx::rdTMemo(char * * TMtextpp, unsigned int cbsize) , in new module TDOSLY17.CPP. 'rdTMemo( ) doesn't really need to be a class TDosLynx member function. But, making it one provides something of a simplification. Adding it to DosLynx mainly amounted to adding its prototype to the definition of class TDosLynx, in TDOSLYNX.H. TDosLynx::rdTMemo( )'s main activities consist of: - Check parameters. - Allocate a new TFileDialog object and file reading buffer. - Run the TFileDialog to get the user's file name input. - open( ) and read( ) the requested file. Append the file's data to any existing string in the given buffer. . . . Until eof or an exception. - Report to the user (via the Messages window). It is easy to call 'rdTMemo( ) from near the end of 'edTArea( ). Once 'rdTMemo( ) returns, 'edTArea( ) updates the document view by means of its call to TURLView::setTArea( ), in TURLVI42.CPP, as usual. In spite of that call, the Messages window, opened by TDosLynx::rdTMemo( )'s report, retains focus for the user's review. The user is then free to dismiss the Messages window (by pressing Esc) and reenter the Textarea Input dialog, or not, as they choose. This allows the Textarea's content to be built up from multiple file reads. And, to be edited by the user, afterwards. Arrangements for dealing with memory limitations, in the DosLynx Real Mode version, haven't turned out nearly as nicely, for the Textarea Input Form control. We don't want to hinder file reading, as long as the needed buffer can be allocated. But, at least one more large memory object has to be allocated in the course of the call to 'setTArea( ). So, the DosLynx Real Mode version's user has to exercise some restraint. By trying not to Paste a file that is too long to be accommodated by the complete process. Needless to say, I have given the new Paste File button a lot of testing. In many of the tests, I used a DosLynx Real Mode version session starting with about 250 KB of free heap memory. I found that this environment could fairly reliably accommodate a text file up to about 32 KB long. However, attempting to reenter a Textarea Input dialog was unreliable, with a Form control containing that much text. That was because 'edTArea( ) was trying to allocate an edit buffer 64 KB long, once a Form control contained anything more than about 16 KB of text. I improved that situation by revising TURLView::edTArea( )'s edit buffer allocation calculation. It now chooses an edit buffer length equal to the present text's length plus (only) about 4 KB. For 'mailDeveloper( ), the user interface considerations of editing cycle and buffer sizes are quite different. First of all, this is because the Send Mail dialog provides an editing buffer that is limited to a total size of only about 4 KB. This limitation is intended to make it very unlikely for the DosLynx Real Mode version to run out of memory while the user is composing e-mail. On the other hand, there isn't any natural way for the Send Mail dialog to be reentered, after Pasting in a File. I decided to follow TDosLynx::mailDeveloper( )'s 'rdTMemo( ) call with a messageBox( ) call, which provides a simple pause dialog, and a direct return to the underlying TMemo dialog. This arrangement seems a little clunky. But, the pause gives the user plenty of time to study the message(s) produced by 'rdTMemo( ). (And, a small part of one document presentation window, behind the Messages and Confirm (pause) windows.) Upon responding to the pause, the user is taken right back to the Send Mail TMemo dialog. So, the main limitation of this arrangement is that the user remains "locked in" once they start the Send Mail dialog. To leave it, they must Send their note or Cancel it. SMTP AUTH LOGIN Support Lacking SMTP AUTH LOGIN support, the DosLynx e-mail client has been of little use for most users. So, this addition was long overdue. I decided to stick with the approach used in PMSMTP. See: http://users.ohiohills.com/fmacall/PMSMTP.TXT for more information. The essential components of the added support are: - A pair of character pointers to base64 encoded e-mail user ID and password strings. These are cp_b64usrid and cp_b64passw. Their definitions have been added to GLOBALS.CPP and GLOBALS.H. They are initialized to zero. - Configuration processing sections for accepting e-mail user ID and password configuration. These have been added to TDosLynx::EvalConfigFile( ), in TDOSLY14.CPP. They may update cp_b64usrid and/or cp_b64passw to point to given base64 encoded string(s). - A command line parameter processing section providing for an alternative e-mail password specification. This has been added to TDosLynx::EvalOption( ), in TDOSLY12.CPP. This may update cp_b64passw to point to a base64 encoded string given in a /S command line option. - A sequence for performing SMTP AUTH LOGIN during e-mail transmission. This has been added to TDosLynx::mailDeveloper( ), in TDOSLY15.CPP. This sequence goes into action when both of the new cp_b64usrid and cp_b64passw variables point to non-empty strings. For further details, see PMSMTP.TXT. The sample DOSLYNX.CFG file provided in the DosLynx v0.31b release package includes a new page of comments on the new b64usrid= and b64passw= configuration values. The README.HTM and README.TXT documents, also provided in the release package, include several new paragraphs describing the new /S command line option. That provides an alternative to b64passw= configuration. The release package's sample DOSLYNX.BA_ file now includes a sample /S command line option. PMSMTP.TXT provides a detailed procedure for the needed base64 encoding of one's e-mail user ID and password strings. Remember to guard all materials that include your e-mail password, whether in an encoded or unencoded form, against disclosure. Another Bout of tcp_ProcessData( ) Trouble The history.txt document for DosLynx v0.28b contains a long story all about heap corruption that was found to be coming from tcp_ProcessData( ), in PCTCP.C. Look for the heading: Another Case of Old Time Heap Corruption. It's a story I'll never forget. That heap corruption was first recognized while visiting http://news.com.com . The problem was fixed by adding checking for tcp_ProcessData( )'s ldiff and diff calculations. Along the way, breakout( ) calls were added to several other PCTCP.C functions, including tcp_read( ). These calls only get made if a situation where heap corruption is about to occur should be detected. Well, wouldn't you know it? When tcp_read( )'s breakout( ) call got made, again recently, that came while visiting http://www.3com.com ! (Those 'com.com sites seem to possess an awesome power!) Like news.com.com, in the earlier story, www.3com.com provided another trigger for fairly readily reproduced trouble. Again, the trouble seemed to be coming from bad values in frag[0] and frag[1]. This time, I pretty quickly zeroed my suspicions in on tcp_ProcessData( ). But, this time, I didn't find a clear source for the trouble. All I could find were a number of things that still bothered me in tcp_ProcessData( )'s (ldiff < 0) clauses. Things such as some strange cases left unchecked. Arithmetic expressions sometimes involving a mix of int, unsigned int, and long int values. And, two min( ) calls. Throughout the (ldiff < 0) clauses: I've added checking. Simplified some of the arithmetic. Eliminated all but simple mixes of long and short int variables in expressions. And, replaced the min( ) calls. As a sample of my revisions, consider the if clause leading off the section I worked on. It was listed in the history.txt for DosLynx v0.28b, after its last extension. I've extended it again and revised some of the subsequent half dozen lines to read as follows: if ((ldiff < 0) && (!(ldiff + lrgdiff)) && (s->rdatalen >= 0) && (diff < (s->maxrdatalen - s->rdatalen)) && (len > 0)) { /* S. Lawson - No out-of-sequence processing of FIN flag. */ * flagsp &= ~tcp_FlagFIN ; diff += s->rdatalen ; /* diff = offset for movmem( )s. */ /* S. Lawson - Handle one dropped segment. */ if (!(s->frag[0])) { /* Just dropped a segment. */ if (len > (x = (s->maxrdatalen - diff))) len = x ; These changes seem to have taken away www.3com.com's ability to trigger a breakout( ) call in tcp_read( ). What I saw for a while after making these changes, at about the previous frequency of the breakout( ) calls, was a GET request response that didn't get started. That is, www.3com.com seemed to be sending several packets that DosLynx didn't seem to accept. Then, DosLynx would report: WWW Alert: Unexpected network read error on response. Some retrys of a troublesome www.3com.com URL obtained a repeated error. But usually, after a ten to ninety second wait, a retry would bring DosLynx something it could accept and successfully present. Once a troublesome www.3com.com URL had been presented successfully, it didn't seem to cause trouble again. For at least a few days. After a few days of testing, with the revised tcp_ProcessData( ), I wasn't able to find any more trouble at www.3com.com! Have I been seeing some kind of cache miss phenomenon arising in the www.3com.com server(s)? I wonder if there are any other 'com.com sites that I should be testing DosLynx with? Updated Score As explained under the heading: Keeping Score, in the history.txt for DosLynx v0.20b, I am maintaining an informal written list of DosLynx issues and their resolutions. In addition to the issues it's always had, this list now includes issues that have come along in connection with the DosLynx Protected Mode version. At the time of this writing, for the release of DosLynx v0.31b (in April 2006), my list has about 461 issues. Of these, about 333 (well over two thirds) have been resolved or ameliorated. This leaves a total of about 128 issues pending. If you will look ahead to the next section for a moment, you may note that several of the pending issues include the parenthetical observation: "This is hardly an issue with the DosLynx Protected Mode version." As far as users of the DosLynx Protected Mode version are concerned, the issues to which this observation applies aren't really pending issues. On the other hand, the DosLynx Protected Mode version does have a few issues, of its own, that don't apply to the Real Mode version. A study identifying issues that don't apply to both versions finds about 121 issues pending for each version, individually. In the next section, I'll list a few of the 128 pending issues that haven't been mentioned above. To Do With the DosLynx Protected Mode version well on its way, pressure is building to implement some of the enhancements listed in the top five entries in the following list. I hope to be able to do something about one or more of these "big five" issues in the next year. Here is a summary of some of the pending issues not discussed anywhere above: - Provide https/SSL support. (This is expected by many Web servers that support data entry.) - Provide cookies support. (This, too, may be expected for data entry.) - Provide Login (original AUTHINFO) support for the news client? - Add an HTTP-Referrer line to the HTTP GET command? - Support the HTML Form