In Development or Planned
This is an update for Semware's debug.si macro:
- In two places fixed the bug, that prev_dbg_fn was compared to dbg_fn case-sensitively.
- Added the "-k" option to just keep using the debugged macro's breakpoints and global variables across calls without the ReloadMenu popping up.
Also included are two test macros "Test1" and "Test2". Test1 calls Test2 three times with the debugger and the new "-k" option. The new debugger is no longer confused by that it internally changed the case of Test2's drive, path and name, and it no longer initializes Test2's global variables for each call.
Inspired by a user thinking out loud about adding an advanced help feature to their tool, I came up with a way to possibly do this with TSE's own Help capabilities.
I think it will be possible to make TSE temporarily substitute TSE's own help by user help. The user help can be provided by a file in a manageable text format.
The result should have almost all the capabilities of a built-in TSE Help page, as well as its advantages and disadvantages.
The tool's work name is "sHelp". This originally stood for "simple help" to distinguish it from the unfinished eHelp tool. However, if sHelp's creation means the eHelp tool will remain unfinished, as currently seems likely, then the sHelp name might need rethinking.
The main differences between the old eHelp and the new sHelp are:
- eHelp was to be integrated with TSE's built-in help, while sHelp is a stand-alone tool that only uses TSE's help engine and a TSE-built-in-help-compatible help text format, and has no integration with TSE's built-in help files.
-
eHelp was to support both TSE's built-in help and
multiple user help files concurrently,
allowing to navigate unlimited help files for the same topic.
sHelp will only support one independent help file at a time, which cannot be TSE's built-in help itself, though it could be a copy thereof. -
eHelp's goal was extending TSE's built-in help.
sHelp's goal is supporting one independent help file at a time for any tool.
I want this tool too, both so I that can add TSE-like help to my
own tools, and to create a one-time copy of TSE's built-in help
that can then be maintained.
A personally greatly satisfying benefit is, that I can put to use
all the knowledge that I gained and most of the tooling that I
created for the unfinished eHelp project.
Done:
- A better proof of concept, that makes the help file's links work internally.
- Made it work as a TSE external help system.
- Copied eHelp code to sHelp. This now compiles and runs with errors that are minor for this stage.
Planned high-level steps:
- A prototype (A publishable impression, not fully functional).
- A minimum viable product (The very basics, usable but limited).
- Better usable versions.
7 Mar 2025:
Progress is still going well.
A lot of eHelp code was copied to sHelp,
and today reached the state
"it compiles and runs, but with errors".
Remaining low-level steps towards a prototype:
- Position the cursor and window correctly.
- Display the correct help file name.
- Mitigate other minor flaws that come with implementing it as a TSE external help system.
- Give a warning for the not-yet-implemented helpline keys and some common help keys. Remaining not-implemented help keys are unmodified, as they are irrelevant for a prototype and allowed to malfunction.
- Create an shelp.syn file.
- Review and tweak the result to make it publishable.
This tool shows a selectable font and its settings in a 16 x 16 character table.
The characters are shown as TSE shows them based on TSE settings.
The TSE session is temporarily shrunk to simplify a side-by-side comparison of different fonts and/or settings using 2 or more TSE sessions.
To do:
- Add font size as a selectable setting.
- Above the table, instead of the font and settings we chose, show the font and settings that Windows actually set based on our request. Or both.
9 Nov 2024 a user made a request, that I interpret as that eList should not always start at line 1, but instead on a list line based on the source's current line, like a search with the "v" option does.
v1.0.1.1
As requested eList now starts at the source file's current line.
When "zooming in" on a longer search string by typing a character, the
list's new current line is set at or as close as possible above the list's
previous current line.
It no longer uses the clipboard, which reduces memory usage,
increases speed, and leaves the clipboard intact.
v1.0.1.2
Disabled the main menu and help line during eList, which loses us nothing
and gains us an extra line of list space
Added numerous progress indicators, useful for processing big buffers.
Emptied the temporary helper buffers when finished to
improve memory usage.
v1.0.1.3
Two huge speed optimizations:
- A search will stop and restart when you type another character.
- When the search string is empty or matches an empty string, the source buffer itself is listed instead of making and listing a copy.
v1.0.1.4
Fixed the major bug in v1.0.1.3, that when eList had just been started and
the first typed character was <Escape>, then the current buffer became
empty.
While editing, the Git macro sporadically makes TSE ignore typed keys.
This is a very elusive and hard to confirm bug, because it almost never occurs.
My main suspect is the Git macro updating the git status
of the current buffer.
My suspicion is, that the macro's
Dos('git.exe … status -s …', _RUN_DETACHED_|…)
call makes TSE ignore keyboard input for a fraction of a second.
My first goal is to find a reliable way to reproduce the bug.
Aside, thinking ahead, I found this way out of the box
possible solution/work-around, that also might make Git usable
in the Console variant of TSE as well.
Currently Git "works" in the Console variant of TSE,
but with user-unfriendly updates of the current buffer's
git status.
7 Nov 2024 a user reported that Uniview sometimes only displays a too short part of the front of a line.
Tests confirm, that this typically happens for longer lines with lots of "Unicode characters".
A review of the Uniview extension makes me want to do a structural improvement of the implementation and documentation.
The upcoming eHelp project would also benefit from this, because the plan is to use UTF-8 for the new help file format.
Math
I found my excuse for writing a Math macro that can do floating point operations: I will need it for the planned Picture macro to scale pictures.
To make optimal use of TSE's native capabilities and limitations, it makes sense to limit a floating point number to MAXSTRINGLEN, MAXLINELEN, or a TSE buffer size (255, 32000, or some value below 3.9 GB).
I cannot think of an excuse to make it longer than MAXSTRINGLEN, and the longer lengths would decrease calculation speed per digit. Tell me if you do know of such an excuse.
Alternatively, for others, if you do not mind installing new executables in Windows, then there is this package from Eckhard Hillmann on Semware's website: FppPack_1_00.zip .
From reading its documentation I gather it provides high-level functionality in TSE, like a calculator-like expression-evaluator shell and a (block?) sum function, and that it provides basic math operators and functions by calling (closed-source) executables from the Windows command prompt.
This will be a tool and extension that lets us open a representation of a picture in TSE.
On 14 Dec 2024 I published XmasGift , an experiment in showing a picture in TSE.
A little more research showed how to show a representation of "any" picture in "any" version of any TSE variant.
The use of the word "representation" is very deliberate.
On the one hand Windows GUI TSE v4.2 upwards can typically show hundreds by hundreds of pixels in 16,777,216 colors.
On the other hand it might be even more actually useful what this
extension can do in a non-graphical Linux server/terminal,
in which Picture will at most be able to show a
picture as 80 by 25 pixels in 16 colors or less.
Even an impression of a picture could be real added value
in such an environment.
Nota bene: This extension has no functionality yet, that is not already present in the Unicode extension.
Done:
- Identify a opened file's Unicode encoding if it has a byte order mark (which is rare).
- Simplisticly display the result with a message.
Todo:
- Identify an opened file's Unicode encoding or code page for all other cases.
- Display a pretty result somewhere.
- Give a signal when the Unicode encoding / the code page does not match the user's default.
- Make it configurable which signal types are used.
Research into 1-byte code pages suggests to me, that it is probably possible to in a useful way detect the three most used ones:
- Code page 437: Windows prompt, English (United States).
- Code page 850: Windows prompt, Other.
- Code page 1252: Windows GUI, All. (a.k.a. "ANSI")
Code pages exist for Unicode too:
- Code page 65001: UTF-8
- Code page 1200: UTF-16LE
- Code page 1201: UTF-16BE
- Code page 12000: UTF-32LE
- Code page 12001: UTF-32BE
Tentative conclusion:
One separate "code page detector", that detects both Unicode
and the three most used 1-byte code pages, seems like a good plan.
This limited functionality should work in all TSE variants.
UniConv will be a newly written ASCII/ANSI/Unicode conversion tool.
It will have none of the interactive and responsive capabilities of the Unicode extension, and will purely focus on the conversion of character encodings.
If it can do a file-to-file conversions without loading a file into TSE, then it will be able to do much larger conversions, and users will be able to load much larger Unicode files. That does require a rewrite of the old code from Unicode.
Deprived from screen interaction, it would be nice if it could be started in these ways:
- From TSE as a macro to convert a string, (line?,) buffer, or file.
- From the command line as a macro.
- From the command line as an executable.
There will exist a critical macro load order dependency between the existing Unicode extension and the future BrowseMode extension below.
(Henceforth in short referred to as Unicode and BrowseMode.)
This requires Unicode to be updated to do its part in enforcing adherence to this dependency.
To be able to test this well, I am first going to try to finally fix the following technical debt.
Unicode is too big to be debugged in TSE's macro debugger.
(TSE's macro debugger starts but malfunctions for too big macros.)
Considered Unicode optimizations:
-
Obsolete code:
Unicode contains flawed, default disabled macro code for displaying the current character's real Unicode form in the the top-left corner of the screen.
This is a remnent from before UniView extisted.
Hopefully this can simply be deleted.
-
UniList:
Unicode contains macro code for displaying a list of all Unicode characters, for optionally selecting one, and for inserting it into the text.
Hopefully this can be split-off into a separate macro, perhaps named UniList.
-
UniClip:
Unicode contains macro code for copying, cutting and pasting TSE ANSI text to and from a Windows Unicode clipboard.
Hopefully this can be split-off into a separate macro, perhaps named UniClip.
-
UniConv:
Unicode contains macro code for displaying Unicode text in an ANSI format, and for converting texts between ASCII, ANSI, and the major Unicode formats.
For a long time I have wished to split the conversion macro code off into a separate macro, perhaps named UniConv.
Hmm, now that I am thinking about this, there are a lot of arguments in favor of UniConv:- It would reduce Unicode's macro code.
- Its use could be shared by Unicode and UniClip.
- I already have other macros that currently use macro code copied from Unicode: They too could call UniConv instead.
- UniConv could be given the capability to convert "unlimited" file sizes.
-
TSE could edit much larger Unicode files for two reasons:
- Currently Unicode uses at least twice a file's size in memory for the conversion.
- If they mostly contain Western-European languages, then UTF-16 and UTF-32 files will shrink 2 and 4 times respectively when converted to TSE's ANSI.
23 Oct 2024 a user published their wish list for TSE, which included a wish for a "Write protected edit error".
I already needed to update my old "BrowsMod" extension, which helps to fulfill that wish.
I decided to completely rewrite my old "BrowsMod" extension into a new "BrowseMode" extension.
The old "BrowsMod" extension can still be found on Semware's "Classic" website.
The new "BrowseMode" will extend the old capabilities and add a new one.
The extended capabilities are, that it will automatically activate TSE's browse mode once for:
- Read-only files.
-
Buffers with names containing user-configured strings.
You can use this to automatically put an opened file in browse mode if its name contains a certain string.
For example, if a file exists in a certain directory, or on a certain drive or network share.
The new capability regards the following.
-
In browse mode TSE itself often already gives a warning
if you try to use a function that would change text,
for example duplicating a line or pasting a block.
However, in browse mode TSE does nothing if you simply type text.
The new extension can give you three types of signal for typing in browse mode: A sound, a shortly shown pop-up message, or a warning that needs to be acknowledged.
Status 28 Nov 2024:
- All stated functionality has been programmed.
-
I dare not publish a provisional version
until I have also added enforcement
of the macro load order of BrowseMode and Unicode.
A wrong macro load order opens the door for a not-attentive user to save a mutilated Unicode file.
For example, if BrowseMode sets the browse mode for a Unicode file before Unicode can convert it to ANSI, then a mutilated file is loaded into TSE, and it would be horrible if the user would save it. - Macro load order enforcement between BrowseMode and Unicode requires that Unicode will be improved too.
-
There will be the minor annoyance,
that executing BrowseMode via the "Execute..." menu
in order to configure it will also change the macro load
order of BrowseMode and Unicode.
This will disable BrowseMode until TSE is restarted, which will restore the macro load order.
A solution of sorts that I found, is to offer a public "BrowseMode -> setup" procedure, like the "cuamark" macro has, which can be accessed via TSE's "Macro -> Show Loaded" menu, and which does not require a TSE restart afterwards. - I want to add the browse mode optimization, that for a group of rapidly typed characters just one signal per configured signal-type is enough.
Linux TSE v4.50 rc 18 upwards (not expandable).
"DirList41" is the development version of the future "DirList" v4.1.
New so far:
-
In Windows DirList is now default independent of its environment's language settings.
This is achieved by retrieving link targets with Windows functions instead of dir commands.
This is a major step forward, even if it did create new loose ends.
Example loose ends:
- Link targets starting with weird prefixes.
- The Windows functions and the dir command each do not retrieve some of the other's details. The now default Windows functions do not retrieve some details of the link type. So I will still have to apply the dir command too for those languages for which the dir output can be interpreted. -
Gained a small but significant speed optimization by optimizing BigInt.
The access status column is now an access error column. Therefore when there is no error, "ok" is replaced by "-". When sorting this sorts all access errors together.
The "no link" indicator was changed from "." to "-". It subjectively looks better, and "-" is in my TSE WordSet, which helps accessing the column.
DirList's speed was increased again by only retrieving link targets for the link types that have them: junctions and symbolic links.
The --alllinktargets parameter generates the old list.
The --folowlinks parameter makes DirList list the child objects of directory links again. This is not recommended. -
Links are no longer followed.
This greatly reduces the run time and list size, and linked subdirectories are no longer counted multiple times in a common parent's directory size.
The following of links will later become a non-default option. - The link type column was condensed to be smaller.
- The column for TSE accessility is now a string indicating a reason.
- A link indicator (Linux) or type (Windows) column was added. Windows has a lot of link types.
- Windows: For junctions and symbolic links the link type and the link's target is now shown. It is shown between square brackets after the name, like "dir /al" does.
Still todo for DirList v4.1:
- Lots of (hopefully small) loose ends.
This considered extension will replace, change and extend TSE's WordWrap functionality.
Summary of main differences with TSE's wordwrapping:
- A paragraph can also be indicated by an indented first line, a bullet point, an enumerator, a multi-line comment, a block of consecutive single-line comments, and HTML tags.
- Wordwrap scope and state is maintaned per buffer.
- Default wordwrapping is OFF.
- When turned ON it keeps wrapping the current paragraph, and turns OFF again when the cursor leaves it.
- A broader scope can be specified by the user, but then other paragraphs in the same buffer only start being wordwrapped when the user changes them.
- Right margin "0" means "the window width".
Detailed points being considered:
- The type of wordwrapping will depend on whether the type of the current buffer is a "text buffer" or a "program/data buffer".
- Here a "text buffer" means a buffer with an extension for which no comment-syntaxhiliting is defined, and all other buffers are of type "program/data buffer".
-
For a "program/data buffer":
- Wordwrapping is default off.
-
The user can toggle wordwrapping between these two values:
- Off: No wordwrapping is done.
- On: If the cursor is in a wordwrappable block, then the block is wordwrapped and keeps being wordwrapped until the cursor leaves the block.
- A wordwrappable block is a multi-line comment, a block of consecutive single-line comments, or text between HTML tags.
- Instead of "On" the extension could show the type of wordwrappable block it recognizes.
-
For a "text buffer":
- Wordwrapping is default off, unless persistence "File" applies.
-
The user can cycle wordwrapping through these persistences:
- None: No wordwrapping is done.
- Paragraph: The current paragraph is wordwrapped and keeps being wordwrapped until the cursor leaves the paragraph.
- Buffer: As "Paragraph" plus the same for other paragraphs in this buffer. For switched-to paragraphs wordwrapping starts when the user makes a change.
- File: As "Buffer" plus for this file it persists across TSE sessions.
-
Wordwrapping will honor TSE's left and right margin
settings.
When the right margin is 0, wordwrapping will use the width of the editing window.
-
A paragraph is a piece of text in a "text buffer" delimited by:
- An empty line.
- An indented line at the start of the paragraph.
- The start of a bullet point.
- The end of a bullet point.
- Bullet points can have levels: A bullet point can contain bullet points.
-
Wordwrapping "text between HTML tags" will only be done for
text between tags that contains no other tags,
other than a few excepted tags.
Such excepted tags: <strong>, <em>, <br>.
<br> will act as a wordwrap delimiter.
Text between <pre> tags will not be wordwrapped. - The extension will show TSE's "W" indicator in the status bar when a block or paragraph is actively being wordwrapped.
-
Probably: Recognize other tag extensions than ".html" and ".htm".
It is not uncommon to encounter .xml files where the whole file is in 1 line. While the xml structure would need to be reformatted by another program, it might become this program's task to wrap the lines inside tags. - Maybe: If the "Status" extension is installed, then it could (optionally) be made to show which type of wordwrapping is active.
- Maybe: Change the background color for an actively being wordwrapped paragraph or block.
Notes:
-
Semware and I mainly use a "-" as a bullet point.
Other seen bullet points are "*", "+" and "ú".
The "ú" is shown as a bullet ("∙") in some OEM character sets. TSE's Help is shown using the system's OEM character set.
In Windows GUI TSE's ANSI character set, characters 149 ("•") and 176 ("°") could be bullet characters too. - I use bullet points inside comments, so it makes sense to me that there they should be wordwrapped too.
- The text of the "Expr" macro has two different occurrences of when to ignore "bullet points".
-
Examples of how to wordwrap bullet points.
Not like this:
Groceries: - The red laundry detergent
but like this:
Groceries:
- The red laundry detergent
And not like this:
Groceries:
- The red laundry
detergent
but like this:
Groceries:
- The red laundry
detergent
(Are my clothes real? Do I want to find out and take the red laundry detergent, or belay my suspicions and take the blue one?) -
I would like to recognize and support enumerators.
For example "1", "1.1", "a.", "a}", "(1)", etc.
But probably not ones that use roman numerals.
Note that enumerators either have their own line or prefix the first line of a paragraph.
Enumerators impact wordwrapping in 2 ways:
An enumerator is a paragraph delimiter.
If an enumerator prefixes the first line of a paragraph, then it counts as whitespace regarding its paragraph's indentation. -
Importing and exporting wrapped text.
Text is wrapped per paragraph.
Paragraph recognition differs between TSE and Word, for example.
In TSE a paragraph is recognized by a blank line or by an indented or outdented first line. This is an existing setting.
In Word a paragraph is recognized by "the enter key".
If you copy a text from Word to TSE, then each Word paragraph becomes a single line.
This implies, that also functions are required to wrap/unwrap a buffer or block of text that was/will be imported/exported from/to some other applications.
Before restarting on eHelp, I am going to investigate
if it makes more sense to expand sHelp instead.
If it does, then all of the below becomes obsolete.
My old ultimate goal of extending TSE's built-in help was postponed on 15 OCt 2023, because it turned too big and time-consuming to continue to pursue.
The new intermediate goal is an up-to-date TSE help document that we can use instead of TSE's not-up-to-date help.
The intermediate help document's detailed goals are, that it:
- Is backward compatible with changes that Semware makes in TSE's help.
- Is forward compatible with the ultimate goal.
- Is simple to build.
- Is easy to maintain.
- Contains TSE's full built-in help, but updated with improvements and extended with missing information.
- Can be used instead of TSE's built-in help.
- Maybe has its own compressed-view search option.
-
Maybe be installable locally.
On the one hand a local HTML file would make it work without an internet connection and increase access speed, on the other hand it would hamper working with its latest up-to-date version. I could create auto-update functionality, but that would cross a line I have not crossed yet.
With the exception of content copied unmodified from Semware's
TSE help, I will be the sole author of the new help
documentation.
Of course I will gladly hear suggestions for its further
improvement.
I intend to make the procedure and its tools and data for creating and maintaining such a new help page publicly available, so that anyone can create their own one if they do so desire.
Future maintenance procedure:
-
Pre-condition:
I have a .ehelp text file (UTF-8) that only contains help topics that differ from or do not exist as TSE's built-in help topics. - Convert Semware's two binary .hlp files to one .ehelp file using the existing eHelp_Hlp2eHelp tool.
- Convert the Semware .ehelp file's code page 437 character-encoding to UTF-8 character-encoding using the existing eHelp_cp437to65001 tool.
- Compare the new Semware .ehelp file to its previous version, for example using CmpBuffers, and if Semware made changes in help topics that also occur as help topics in my .ehelp file, then manually apply those changes to my .ehelp file.
- Create a combined .ehelp file that contains Semware's help topics overwritten with my help topics using a to-be-built tool.
- Convert the combined .ehelp file to a .html file using a to-be-built tool.
- Publish the .html file.
Version 1 to-do list (10 Jan 2025):
- Done: Create a tool to convert code page 437 to UTF-8.
- Create a combine-tool, that creates a combined .ehelp file that contains Semware's help topics overwritten with my help topics.
- Create a tool to convert a .ehelp file to a .html file.
Version 1.x maybe-to-do list:
- Create a fast access method from TSE.
- Maybe make the .html file locally available.
- Maybe make the local version auto-updatable.
- Maybe create a compressed-view search function.
