Convert Bytes to String in Python: A Tutorial for Newbies -

Picture by Creator

In Python, strings are immutable sequences of characters which can be human-readable and usually encoded in a particular character encoding, reminiscent of UTF-8. Whereas bytes symbolize uncooked binary information. A byte object is immutable and consists of an array of bytes (8-bit values). In Python 3, string literals are Unicode by default, whereas byte literals are prefixed with a b.

Changing bytes to strings is a typical process in Python, notably when working with information from community operations, file I/O, or responses from sure APIs. It is a tutorial on the best way to convert bytes to strings in Python.

1. Convert Bytes to String Utilizing the decode() Technique

Essentially the most simple method to convert bytes to a string is utilizing the decode() methodology on the byte object (or the byte string). This methodology requires specifying the character encoding used.

Observe: Strings wouldn’t have an related binary encoding and bytes wouldn’t have an related textual content encoding. To transform bytes to string, you should use the decode() methodology on the bytes object. And to transform string to bytes, you should use the encode() methodology on the string. In both case, specify the encoding for use.

Instance 1: UTF-8 Encoding

Right here we convert byte_data to a UTF-8-encoded string utilizing the decode() methodology:

# Pattern byte object
byte_data = b'Hi there, World!'

# Changing bytes to string 
string_data = byte_data.decode('utf-8')

print(string_data)

You must get the next output:

You may confirm the info varieties earlier than and after the conversion like so:

print(kind(bytes_data))
print(kind(string_data))

The info varieties must be as anticipated:

Output >>>
<class 'bytes'>
<class 'str'>

Instance 2: Dealing with Different Encodings

Typically, the bytes sequence could comprise encodings apart from UTF-8. You may deal with this by specifying the corresponding encoding scheme used once you name the decode() methodology on the bytes object.

Right here’s how one can decode a byte string with UTF-16 encoding:

# Pattern byte object 
byte_data_utf16 = b'xffxfeHx00ex00lx00lx00ox00,x00 x00Wx00ox00rx00lx00dx00!x00'

# Changing bytes to string 
string_data_utf16 = byte_data_utf16.decode('utf-16')

print(string_data_utf16)

And right here’s the output:

Utilizing Chardet to Detect Encoding

In follow, you could not all the time know the encoding scheme used. And mismatched encodings can result in errors or garbled textual content. So how do you get round this?

You need to use the chardet library (set up chardet utilizing pip: pip set up chardet) to detect the encoding. After which use it within the `decode()` methodology name. Right here’s an instance:

import chardet

# Pattern byte object with unknown encoding
byte_data_unknown = b'xe4xbdxa0xe5xa5xbd'

# Detecting the encoding
detected_encoding = chardet.detect(byte_data_unknown)
encoding = detected_encoding['encoding']
print(encoding)

# Changing bytes to string utilizing detected encoding
string_data_unknown = byte_data_unknown.decode(encoding)

print(string_data_unknown)

You must get an analogous output:

Error Dealing with in Decoding

The bytes object that you just’re working with could not all the time be legitimate; it could typically comprise invalid sequences for the required encoding. It will result in errors.

Right here, byte_data_invalid
comprises the invalid sequence xff:

# Pattern byte object with invalid sequence for UTF-8
byte_data_invalid = b'Hi there, World!xff'

# strive changing bytes to string 
string_data = byte_data_invalid.decode('utf-8')

print(string_data)

Whenever you attempt to decode it, you’ll get the next error:

Traceback (most up-to-date name final):
  File "/residence/balapriya/bytes2str/fundamental.py", line 5, in 
	string_data = byte_data_invalid.decode('utf-8')
              	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec cannot decode byte 0xff in place 13: invalid begin byte

However there are a few methods you’ll be able to deal with these errors. You may ignore such errors when decoding or you’ll be able to substitute invalid sequences with a placeholder.

Ignoring Errors

To disregard invalid sequences when decoding, you’ll be able to set the errors you’ll be able to set errors to ignore within the decode() methodology name:

# Pattern byte object with invalid sequence for UTF-8
byte_data_invalid = b'Hi there, World!xff'

# Changing bytes to string whereas ignoring errors
string_data = byte_data_invalid.decode('utf-8', errors="ignore")

print(string_data)

You’ll now get the next output with none errors:

Changing Errors

You may as properly substitute invalid sequences with the placeholder. To do that, you’ll be able to set errors to substitute as proven:

# Pattern byte object with invalid sequence for UTF-8
byte_data_invalid = b'Hi there, World!xff'

# Changing bytes to string whereas changing errors with a placeholder
string_data_replace = byte_data_invalid.decode('utf-8', errors="substitute")

print(string_data_replace)

Now the invalid sequence (on the finish) is changed by a placeholder:

Output >>>
Hi there, World!�

2. Convert Bytes to String Utilizing the str() Constructor

The decode() methodology is the commonest method to convert bytes to string. However you may also use the str() constructor to get a string from a bytes object. You may cross within the encoding scheme to str() like so:

# Pattern byte object
byte_data = b'Hi there, World!'

# Changing bytes to string
string_data = str(byte_data,'utf-8')

print(string_data)

This outputs:

3. Convert Bytes to String Utilizing the Codecs Module

One more methodology to transform bytes to string in Python is utilizing the decode() perform from the built-in codecs module. This module gives comfort capabilities for encoding and decoding.

You may name the decode() perform with the bytes object and the encoding scheme as proven:

import codecs

# Pattern byte object
byte_data = b'Hi there, World!'

# Changing bytes to string
string_data = codecs.decode(byte_data,'utf-8')

print(string_data)

As anticipated, this additionally outputs:

Abstract

On this tutorial, we realized the best way to convert bytes to strings in Python whereas additionally dealing with totally different encodings and potential errors gracefully. Particularly, we realized the best way to:

Use the decode() methodology to transform bytes to a string, specifying the proper encoding.
Deal with potential decoding errors utilizing the errors parameter with choices like ignore or substitute.
Use the str() constructor to transform a legitimate bytes object to a string.
Use the decode() perform from the codecs module that’s constructed into the Python commonplace library to transform a legitimate bytes object to a string.

Glad coding!

Bala Priya C is a developer and technical author from India. She likes working on the intersection of math, programming, information science, and content material creation. Her areas of curiosity and experience embrace DevOps, information science, and pure language processing. She enjoys studying, writing, coding, and low! At present, she’s engaged on studying and sharing her information with the developer group by authoring tutorials, how-to guides, opinion items, and extra. Bala additionally creates participating useful resource overviews and coding tutorials.

Convert Bytes to String in Python: A Tutorial for Newbies