Intro

Querying and setting a code page

Tested system locales and their code page

Code pages side by side

The following is a simplification, and is limited in its viewpoint by only looking at languages of Western European origin.

People use characters to read and write. Computers use numbers. Therefore people assigned numbers to characters.

Because computers were extremely expensive, the initial trend was to use only 1-byte (8-bit) numbers per character, enforcing a maximum of 256 characters per character set.

For languages that started as Western European, 255 characters were enough for ordinary reading and writing. Other languages are outside the scope of this document.

The first 128 characters of each character set were standardized under the name "ASCII", and are the same for all such languages.

Initially each country created its own character set, in which the last 128 characters differed per country and language. Nowadays, in practice, the huge amount of 1-byte character sets have been reduced to 3 mainly used ones.

A set of characters with numbers assigned to them is also referred to as a code page, and the set is identified by a code page number.
The mainly used 1-byte code pages are:

• 437 English (United States)
  Typically used for the Windows command prompt, .bat and .cmd scripts, and the Console version of TSE.
  Without the need for accented characters, its character set has lots of drawing-characters.
• 850 Other languages with a Western European origin, including English used outside the United States.
  Typically used for the Windows command prompt, .bat and .cmd scripts, and the Console version of TSE.
  Its character set gave up a part of the drawing-characters to be able to provide accented characters for other languages.
• 1252 A Windows standard for languages with a Western European origin.
  Typically used everywhere for Windows GUI programs that use a 1-byte character set, like the GUI version of TSE.
  Its character set gave up all drawing-characters to be able to provide even more non-drawing-characters.
  The code page is informally also referred to as "ANSI".

TSE natively only supports 1-byte character encodings. For multi-byte character encodings it shows multiple "garbage" characters per source character.

The Console variant of TSE uses the code page of its environment for showing edited text. Internally it expects code page 437 when displaying its borders and help screens, with causes a few wrong characters to be shown there in other code pages.

The GUI and Linux variants of TSE use code page 1252, whatever the externally set code page is.

GUI TSE can use extensions to handle Unicode files.

Windows itself used to internally default to code page 1252 (ANSI). and nowadays internally defaults to UTF-16LE. This is why Windows supports "A" and "W" versions of its APIs and data types.

The Windows system locale has a checkmark for "Unicode", which makes cmd command lines use code page 65001 (aka UTF-8).

Windows Office documents internally use UTF-8.

The web mostly uses UTF-8.

"UTF-…" are names for Unicode character encodings. Each Unicode character encoding refers to a different implementation of (read: bytes used for) the same Unicode character set. To refer to a character independent of its encoding, it is ubiquitously referred to by its Unicode administrative number, called a "code point".

Unicode character encodings can also be referred to by a code page number:

Code page number	Unicode name	Bytes per character (*)
65001	UTF-8	1 to 4
1200	UTF-16LE	2 or 4
1201	UTF-16BE	2 or 4
12000	UTF-32LE	4
12001	UTF-32BE	4

(*) depending on the character.

A code page setting is independent of the "Windows display language" setting. For example, you can totally garble a command prompt program's output by trying to make it show a French file that has accented characters while using an US English code page.

In Windows you can change the code page at 2 levels:

• Locally for the current command prompt.
• Globally for the whole computer, which makes it the default for all future command prompts.

Locally:

You can query a command prompt's code page with the command "chcp" without parameters.

You can change a command prompt's code page with the command "chcp <code page number>". For example:
      chcp 437

You cannot change a TSE instance's code page from within that TSE instance for that TSE instance.
You can query it with the TSE macro command "Query(CodePage)".

When you start a .bat or .cmd file or a command from TSE, then you have the option to begin the command (file) with a "chcp <code page number>" command to set the desired character set for the output.

Unfortunately, lots of prompt commands that I typically use ignore the local code page and use the global code page instead, while some of them do adhere to the local code page.

Globally:

You cannot globally change the code page directly.

You have to change the "system locale", which has a "Language (Country)" format, and then Windows secretly sets its global code page based on that.

In Windows 10 you can set the system locale with Settings -> Time&Language -> Language -> Administrative language settings -> Change system locale.

Caveat:
If you check "Beta: Use Unicode UTF-8 for worldwide language support", then this overrules the system locale's code page, and sets the code page to 65001 (Unicode's UTF-8).
This is an interesting option, but be aware of its impact on all command prompts and spawned command files.

Because many system locales share a same code page, often nothing happens when you change the system locale, but when the system locale changes the code page too, then you will be asked to restart Windows to activate the change.

Click a code page's header to view additional info on Wikipedia.

A "code point" is Unicode's administrative number for a character.
It is great for uniquely identifying a character across code pages and Unicode encodings.

NBSP is the "no break space" character.
SHY is the "soft hyphen" character.
"₧" is one character, signifying the "pesetas" currency.