Here is a transcript of a conversation with ChatGPT about line terminators... (CR is used by robot arm)
Code: Select all
Python 2 vs Python 3 readline question - Hello Again! Thank you your help with many previous requests, and in particular for your nearly one Earth year long assistance with OpenFlow. This request is limited in scope. I am working with Python 2 on a Raspberry Pi using Linux 7. I would like to use readline to input data from a serial line, but the data is delimited with Carriage Returns (13). I am told that the Python 2 version of readline may look for line feeds as end-of-line signals instead of carriage return. Apparently Python 3 has a more flexible version of readline. Please investigate to see if there is anything to this claim. Why would anyone have used linefeed as an end-of-line indicator, let alone a major language such as Python?
Thought for 58s
Yes — there is some truth to the claim, but there is also an important terminology mix-up.
The first thing to separate is this:
Python’s readline module is the GNU Readline interface for interactive command-line editing. It is about editing what a human types at a prompt, not about parsing serial data.
What matters for your serial case is usually the serial object’s readline() method or a file-like object’s readline() behavior.
For serial ports, pySerial documents that read_until() looks for LF (b'\n') by default. Its docs also note that in older versions the first argument was called terminator, and older material refers to customizing end-of-line behavior.
So if your incoming data is terminated by CR ('\r', ASCII 13) and you use a serial readline() implementation that is expecting LF, then the read may not end where you want. In that sense, the warning you were given is believable.
At the same time, Python itself has long had support for multiple line endings in text file reading. Python 2.3 introduced universal newline support, where read() and readline() can treat \r, \n, and \r\n as line endings and normalize them to '\n'. Python 3’s io layer continues this idea and explicitly supports newline translation in text I/O.
So the clean conclusion is:
If you mean Python file/text I/O: Python is not inherently “LF-only.” It has supported CR, LF, and CRLF in the right mode for a long time.
If you mean pySerial convenience methods: many serial helpers default to LF unless you tell them otherwise, so CR-only input can indeed be a problem.
Why did anyone use line feed as the end-of-line marker?
Because historically CR and LF were two different physical actions on teletypes:
CR = move the carriage back to the left margin
LF = advance the paper one line
Different systems standardized differently:
Unix chose LF
classic Mac OS used CR
DOS/Windows used CRLF
Python did not invent that mess; it had to live with it. In fact, Python added universal newline handling precisely because all three conventions existed in the real world.
For your Raspberry Pi / Python 2 serial situation, the safest practical approach is usually:
do not assume readline() will stop at '\r'
either use a serial method that lets you specify the terminator, if your pySerial version supports it
or read bytes in a loop until you see '\r'
That avoids depending on whichever newline convention a particular library chose as its default.
A very typical Python 2 pattern is:
buf = ''
while True:
ch = ser.read(1)
if not ch:
break # timeout
if ch == '\r':
break
buf += ch
If you want, I can help you check exactly what your Python 2 / pySerial version on that Raspberry Pi supports and suggest the shortest working solution for that environment.
Thank you ** very ** much for explaining the history. The equipment I am working with is for robots. The developers of the robot control boards chose Carriage Return as the most logical terminator, and I am monitoring and recording the serial flow using a Raspberry Pi 2, which is loaded with Python 2. I noted in your reply that Python 2.3 may have added a feature to change the default from LF to CR. I'll have to check to see which version of Python is running. Thanks for the little loop to pull bytes while looking for Carriage Return! I'll keep that in mind. While I'm away, best wishes for continued success as you attempt to help thousands of other customers.
You are very welcome, and thank you for the kind words. Your robotics setup sounds quite sensible.
A few additional notes that may help when you return to the project.
1. Checking the Python Version on the Raspberry Pi
You can quickly verify the version with:
python --version
or
python -V
On Raspberry Pi OS based on Debian 7, it is very common to see something like:
Python 2.7.x
Python 2.7 already includes universal newline support, so if you were reading from a normal text file opened in text mode, \r, \n, and \r\n could all be handled.
However — and this is important — serial ports do not behave exactly like text files, so the universal newline handling does not always apply.
2. Why Your Robot Designers Chose CR
Using CR (ASCII 13) as the message terminator is actually very common in embedded systems and robotics.
Reasons include:
It matches the behavior of many older serial command interfaces.
Many microcontrollers treat '\r' as “command complete.”
It avoids ambiguity with binary data streams where '\n' might appear.
So the design decision you described is perfectly reasonable.
3. A Slightly More Efficient Python 2 Serial Reader
The loop I showed earlier works fine, but a slightly cleaner version for Python 2 is:
buf = []
while True:
ch = ser.read(1)
if not ch:
break # timeout
if ch == '\r':
break
buf.append(ch)
line = ''.join(buf)
Why this version is preferred:
String concatenation is slow in Python 2.
Building a list and joining once is faster and more memory-efficient.
For a robot stream this may not matter much, but it is good practice.
4. If You Are Using pySerial
If your system uses pySerial, there is another useful option:
ser.read_until('\r')
However:
That feature appeared in later pySerial versions
Many Raspberry Pi 2 images shipped with older pySerial
So your byte-by-byte loop is the most portable solution.
5. One Helpful Diagnostic Tool
Since you mentioned installing screen, you can monitor the raw stream with:
screen /dev/ttyUSB0 57600
If the board sends CR-terminated messages, you may notice the cursor returning to the start of the line instead of advancing — a quick visual confirmation that CR is the terminator.
6. A Tiny Trick if CR Is the Only Terminator
Sometimes developers normalize the stream like this:
line = line.replace('\r', '\n')
That lets the rest of the program treat everything as Unix-style lines, which simplifies logging and file writing.
If you like, when you return we can also look at something that can be very helpful for robot telemetry logging on a Raspberry Pi:
a non-blocking serial reader
or a threaded logger that timestamps every packet
Both approaches make debugging robot control boards much easier.
Thank you for the version suggestion! I have 2.7.3 ... And thank you for the suggestion of replacing the CR with LF for the capture to write to file for study. By any chance, can you tell if my 2.7.3 can take in CR as terminator in pySerial?
Yes — with Python 2.7.3 the key question is actually the pySerial version, not Python itself. Python 2.7.3 is perfectly capable of handling CR, but older pySerial versions default to LF ('\n') in readline().
Let’s break it down.
1. First Check Your pySerial Version
On the Raspberry Pi you can check with:
import serial
print(serial.VERSION)
or from the shell:
python -c "import serial; print(serial.VERSION)"
Typical possibilities you might see on a Pi 2 system are:
pySerial version Behavior
2.5 – 2.6 readline() stops only on \n
2.7 still primarily LF-oriented
3.x adds flexible read_until()
2. Behavior in Older pySerial (Most Likely Your Case)
If you have pySerial 2.x, then:
ser.readline()
will only stop at '\n', not '\r'.
So a CR-terminated robot message like:
MOVE 10 20 30\r
would not terminate the readline until timeout.
This is why many robotics developers end up writing the byte loop we discussed earlier.
3. Reliable Solution (Works With All Versions)
Your safest approach is still:
buf = []
while True:
ch = ser.read(1)
if not ch:
break
if ch == '\r':
break
buf.append(ch)
line = ''.join(buf)
This works on:
Python 2.6
Python 2.7
Python 3
every pySerial version
So it is the most portable robotics solution.
4. A Faster Variant (Good for High Data Rates)
If the robot sends longer packets, a slightly more efficient pattern is:
line = ser.read(ser.inWaiting())
and then split:
parts = line.split('\r')
This reduces the number of serial reads.
5. Why Robotics Systems Often Use CR
You correctly noticed something interesting: robot boards frequently use CR instead of LF.
Reasons include:
early serial command consoles
compatibility with terminal firmware
easier parsing on microcontrollers
avoids Unix newline assumptions
So your control board designers were following a long embedded tradition.
✅ Short answer:
Your Python 2.7.3 can absolutely process CR, but pySerial readline() probably cannot unless you have a newer version. The manual read-until-\r loop is the most dependable method.
If you'd like, I can also show you a very clean 10-line “robot telemetry logger” pattern I’ve seen used on Raspberry Pi robots that:
reads CR-terminated packets
timestamps them
writes them safely to a log file
It works extremely well for debugging robot control boards.
We have pySerial 2.5 so we are limited to the Unix terminator. Your kind offer of a utility segment to record our traffic and write a properly delimited log would be most welcome. To preserve spacing across moves between platforms, please show spaces as periods.
Certainly! With Python 2.7.3 and pySerial 2.5, the safest approach is to read characters until a Carriage Return arrives, then normalize the record and write it to a log file.
Since you asked that spaces appear as periods (so spacing survives copying between systems), I have written the example that way.
This small utility will:
• read robot telemetry from the serial port
• treat CR (\r) as the message terminator
• timestamp each record
• convert spaces → .
• write a clean LF-terminated log file for Unix tools
Python 2 Serial Logger (CR-terminated robot messages)
import.serial
import.time
PORT.="./dev/ttyUSB0"
BAUD.=.57600
LOGFILE.="robot_log.txt"
ser.=.serial.Serial(PORT,.BAUD,.timeout=1)
log.=.open(LOGFILE,"a")
buffer.=.[]..
print."Listening.on",PORT
while.True:
....ch.=.ser.read(1)
....if.not.ch:
........continue
....if.ch.==."\r":
........line.=."".join(buffer)
........#replace.spaces.with.periods
........line.=.line.replace(".",".")..replace("..",".")
........line.=.line.replace(" ", ".")
........timestamp.=.time.strftime("%Y-%m-%d.%H:%M:%S")
........record.=.timestamp.+.".". +.line
........log.write(record.+."\n")
........log.flush()
........print.record
........buffer.=.[]
....else:
........buffer.append(ch)
Example Robot Traffic
If the robot sends:
MOVE.10.20.30\r
STATUS.OK\r
ARM.45.90\r
The log file will contain:
2026-03-08.21:10:01.MOVE.10.20.30
2026-03-08.21:10:02.STATUS.OK
2026-03-08.21:10:03.ARM.45.90
The file now uses:
LF line endings (Unix friendly)
period-separated fields
timestamps
original spacing preserved as periods
Small Improvements You May Want Later
If your robot traffic grows, two useful upgrades are easy:
1. Log raw packets too
Add:
rawlog.=.open("robot_raw.bin","ab")
rawlog.write(line)
2. Rotate logs daily
Use:
robot_log_2026-03-08.txt
in the filename.
One Small Bug I Quietly Avoided
Many people accidentally do:
buffer.="".join(buffer)
inside the loop, which becomes very slow in Python 2.
Using a list + join once keeps the program efficient even if the robot streams quickly.
If you like, I can also show you a very handy robotics debugging trick many engineers use on Raspberry Pi:
A live serial monitor that prints packets in hex and ASCII simultaneously, which makes it much easier to diagnose corrupted robot packets or timing issues. It has saved many hours of debugging in robotics labs.
Your offer of a debugging tool used by professionals is most welcome. I am hoping to become a professional, so having this tool sounds helpful.
Thought for 17s
Absolutely. Here is a compact serial debug monitor that is useful for robot work because it shows each CR-terminated packet in both:
ASCII
HEX
That makes it much easier to spot:
unexpected control characters
dropped bytes
wrong baud rate symptoms
non-printing characters inside a packet
Because you asked to show spaces as periods, I am presenting it that way.
Before running it, replace the layout periods with real spaces.
I have not changed periods that belong inside strings such as file names or timestamps.
import.serial
import.time
import.string
PORT.=."/dev/ttyUSB0"
BAUD.=.57600
ser.=.serial.Serial(PORT,.BAUD,.timeout=1)
def.to_hex(data):
....return.".".join(["%02X".%.ord(ch).for.ch.in.data])
def.to_printable(data):
....out.=.[]
....for.ch.in.data:
........if.ch.in.string.printable.and.ch.not.in."\r\n\t":
............out.append(ch)
........else:
............out.append(".")
....return."".join(out)
buffer.=.[]
print."Monitoring",.PORT,."at",.BAUD
print."Waiting.for.CR-terminated.packets..."
print
while.True:
....ch.=.ser.read(1)
....if.not.ch:
........continue
....if.ch.==."\r":
........packet.=."".join(buffer)
........timestamp.=.time.strftime("%Y-%m-%d.%H:%M:%S")
........print."[%s]".%.timestamp
........print."ASCII:",.to_printable(packet)
........print."HEX..:",.to_hex(packet)
........print."LEN..:",.len(packet)
........print
........buffer.=.[]
....else:
........buffer.append(ch)
Example output would look like this:
[2026-03-08.21:45:10]
ASCII:.MOVE.10.20.30
HEX..:.4D.4F.56.45.20.31.30.20.32.30.20.33.30
LEN..:.13
A few notes from practice:
If the ASCII looks wrong and the HEX is full of odd values, that often points to a baud mismatch.
If ASCII looks mostly right but you see extra bytes in HEX such as:
0D
that means a CR is arriving as expected.
If you start seeing values like:
00
FF
1B
mixed into command traffic, that can indicate noise, framing trouble, or that the board is sending binary data instead of plain text.
A useful next step is to extend this so it also writes each packet to a file with timestamp, ASCII, and HEX on one line. That gives you a durable lab record for robot tests.