Py Notes 4

python regex

Regular expressions (regex) in Python provide a powerful way of searching, matching, and manipulating strings. Python's re module supports regular expression operations. Here is a basic guide on how to use regular expressions in Python:

Importing the `re` Module

First, you need to import Python's re module:

import re

Basic Regex Functions

re.search(pattern, string, flags=0): Searches the string for a match, and returns a match object if there is a match.
re.match(pattern, string, flags=0): Determines if the regex matches at the beginning of the string.
re.findall(pattern, string, flags=0): Finds all non-overlapping matches of the pattern in the string, and returns them as a list.
re.sub(pattern, repl, string, count=0, flags=0): Replaces the matches with a replacement string.

Compiling Regular Expressions

For repeated use of the same pattern, you can compile a regex into a regular expression object, which can then be used for matching:

pattern = re.compile(r'\bfoo\b')

Examples

Finding a Match

import re

text = "Python is fun"
match = re.search(r'Python', text)

if match:
    print("Match found:", match.group())
else:
    print("No match")

Finding All Matches

import re

text = "Python is fun, because Python is easy to learn."
matches = re.findall(r'Python', text)

print("Matches found:", matches)

Replacing Text

import re

text = "Python is fun"
replaced_text = re.sub(r'fun', 'awesome', text)

print(replaced_text)

Flags

The re module supports various flags to modify the behavior of the regex matching, such as re.IGNORECASE (ignore letter case), re.MULTILINE (multi-line matching), and others.

Special Characters and Sequences

Regular expressions use special characters and sequences to denote types of patterns, like:

.: Matches any character except a newline.
^: Matches the start of the string.
$: Matches the end of the string.
*: Matches 0 or more repetitions of the preceding RE.
+: Matches 1 or more repetitions of the preceding RE.
?: Matches 0 or 1 repetition of the preceding RE.
\d: Matches any decimal digit; equivalent to the set [0-9].
\D: Matches any non-digit character; equivalent to [^0-9].
\s: Matches any whitespace character.
\S: Matches any non-whitespace character.
\w: Matches any alphanumeric character; equivalent to [a-zA-Z0-9_].
\W: Matches any non-alphanumeric character; equivalent to [^a-zA-Z0-9_].
[...]: Used to indicate a set of characters.

Regular expressions can be very powerful but may become complex for intricate patterns. It's often helpful to test your regex patterns with a small sample of your data before applying them in your code. Online regex testers can be invaluable for debugging more complex expressions.

list files with specific pattern

To list files in a directory with a specific pattern in Python, you can use the glob module, which provides a function for making file lists from directory wildcard searches. This is especially useful for finding files that match a certain pattern.

Here's a basic example using glob.glob():

import glob

# Pattern to match files - for example, all '.txt' files
pattern = "*.txt"

# List all files matching the pattern
files = glob.glob(pattern)

print(files)

If you're working in a specific directory and want to list files matching a pattern within that directory, you can include the directory path in the pattern:

import glob

# Assuming you're looking for '.txt' files in 'documents' directory
pattern = "documents/*.txt"

# List all files matching the pattern
files = glob.glob(pattern)

print(files)

For recursive searches (to find files that match a pattern in the current directory and all subdirectories), use glob.glob() with the recursive=True argument and the ** pattern:

import glob

# Pattern to match files recursively
pattern = "**/*.txt"

# List all files matching the pattern, including in subdirectories
files = glob.glob(pattern, recursive=True)

print(files)

The glob module is a straightforward way to find files by pattern matching on filenames. However, it performs pattern matching using Unix shell-style wildcards, which are more limited in expressiveness compared to regular expressions. For more complex patterns that require the full power of regular expressions, you might need to combine os.walk() with re module functions to manually filter filenames.

argsparse multiple values for single argument + python

In Python, the argparse module can be configured to accept multiple values for a single argument. This is typically done using the nargs parameter of the add_argument() method. Here are a few common use cases:

Accepting Multiple Values

To specify that an argument should accept a specific number of values, you can set nargs to an integer. For example, to accept exactly three values:

import argparse

parser = argparse.ArgumentParser(description="Example with multiple values for a single argument.")
parser.add_argument('--myarg', nargs=3, help='An argument that accepts exactly three values')

args = parser.parse_args()

print(args.myarg)

Accepting a List of Values

If you want an argument to accept a variable number of values (at least one), you can set nargs to '+'. This is useful when you don't know in advance how many values the argument will receive:

import argparse

parser = argparse.ArgumentParser(description="Example with a list of values for a single argument.")
parser.add_argument('--myarg', nargs='+', help='An argument that accepts a list of values')

args = parser.parse_args()

print(args.myarg)

Accepting Zero or More Values

To allow an argument to accept zero or more values, set nargs to '*'. This allows the argument to be optional, and if provided, it can have any number of values:

import argparse

parser = argparse.ArgumentParser(description="Example with zero or more values for a single argument.")
parser.add_argument('--myarg', nargs='*', help='An argument that accepts zero or more values')

args = parser.parse_args()

print(args.myarg)

Accepting a Pair of Values

If you need an argument to always accept a pair of values (exactly two), set nargs to 2:

import argparse

parser = argparse.ArgumentParser(description="Example with a pair of values for a single argument.")
parser.add_argument('--myarg', nargs=2, help='An argument that accepts a pair of values')

args = parser.parse_args()

print(args.myarg)

When running these scripts from the command line, you would provide the values for --myarg like so:

python script.py --myarg value1 value2 value3

Replace script.py with the name of your script file. The args.myarg will then contain the list of values provided on the command line.

list files with specific pattern python

To list files in a directory that match a specific pattern in Python, you can use the glob module for simple patterns or the os module combined with regular expressions for more complex patterns. Here's how to do it with both methods:

Using the `glob` Module

The glob module allows you to specify a pattern and returns a list of file paths that match the pattern. It supports wildcard characters such as * (matches everything) and ? (matches any single character).

import glob

# List all .txt files in the current directory
txt_files = glob.glob('*.txt')

print(txt_files)

# List all .txt files in a specific directory ("/path/to/directory/")
txt_files_in_directory = glob.glob('/path/to/directory/*.txt')

print(txt_files_in_directory)

# Recursively list all .txt files in the current directory and subdirectories
all_txt_files_recursive = glob.glob('**/*.txt', recursive=True)

print(all_txt_files_recursive)

Using the `os` Module with Regular Expressions

If you need more complex pattern matching that isn't supported by the simple wildcard patterns of glob, you can use the os module to list files in a directory and then filter them using regular expressions from the re module.

import os
import re

# The directory to search files in
directory = '.'

# The pattern to match filenames against
pattern = re.compile(r'^example_.*\.txt$')  # Example pattern: files starting with 'example_' and ending with '.txt'

# List files in the specified directory and filter them based on the pattern
matching_files = [f for f in os.listdir(directory) if pattern.match(f)]

print(matching_files)

In this second method, os.listdir(directory) lists every file and directory in directory, and the list comprehension filters this list to include only the files that match the specified regular expression pattern.

Choose the method that best fits your needs: use glob for simplicity and most common cases, or use os with re for more complex pattern matching requirements.

get numbers in python using regex

To extract numbers from a string in Python using regular expressions, you can use the re module. The findall() function is particularly useful for this task, as it returns all non-overlapping matches of a pattern in a string, as a list of strings. To match numbers, you can use a pattern like \d+, where \d matches any digit, and + means one or more occurrences of the preceding element.

Here's a simple example to extract all numbers from a string:

import re

# Sample string
text = "There are 123 apples, 50 oranges, and 10 bananas."

# Pattern to match numbers
pattern = r'\d+'

# Find all matches of the pattern
numbers = re.findall(pattern, text)

# Convert matched strings to integers
numbers = [int(num) for num in numbers]

print(numbers)

This script will output:

[123, 50, 10]

In this example, re.findall() searches the text for all substrings that match the pattern and returns them as a list of strings. Then, a list comprehension is used to convert each string in the list to an integer.

If you also need to capture floating point numbers, you can adjust the regex pattern. For example, to match both integers and decimal numbers, you can use a pattern like r'\d+(\.\d+)?', which matches sequences of digits that may optionally be followed by a decimal point and more digits:

# Pattern to match integers and decimal numbers
pattern = r'\d+(\.\d+)?'

# Sample string with decimal numbers
text = "The weights are 45.5 kg, 50.2 kg, and 55 kg."

# Find all matches and filter out None
matches = re.findall(pattern, text)
numbers = [float(match[0]) if match[1] else int(match[0]) for match in matches]

print(numbers)

This pattern, r'\d+(\.\d+)?', will match integers as well as decimal numbers, allowing you to extract a wider range of numeric values from a string.

glob.glob with natural sorting + python

Python's glob.glob() function returns a list of file paths matching a specified pattern, which can be very useful for file manipulation and processing tasks. However, glob does not sort the returned file paths in a natural or human-readable order by default. In computing, natural sorting is the ordering of strings in alphabetical order, except that multi-digit numbers are ordered as a single character.

To achieve natural sorting of the file paths returned by glob.glob(), you can use the re module to define a function that extracts the numeric parts of the filenames for comparison. Then, use this function as the key in the sorted() function.

Here's how you can do it:

import glob
import re

def atoi(text):
    return int(text) if text.isdigit() else text

def natural_keys(text):
    '''
    alist.sort(key=natural_keys) sorts in human order
    http://nedbatchelder.com/blog/200712/human_sorting.html
    (See Toothy's implementation in the comments)
    '''
    return [ atoi(c) for c in re.split(r'(\d+)', text) ]

# Path pattern to match files
pattern = "*.txt"

# Get a list of file paths matching the pattern
file_paths = glob.glob(pattern)

# Sort the file paths in natural/human order
sorted_file_paths = sorted(file_paths, key=natural_keys)

print(sorted_file_paths)

In this example, natural_keys is a function that splits the filename into parts of digits and non-digits and converts the digit parts into integers. This allows sorted() to compare these parts in a way that results in natural sorting. The atoi function helps by converting text to integers if possible, assisting in the natural sorting process.

This approach will ensure that file names like file1.txt, file2.txt, ..., file10.txt are sorted in that order, rather than file1.txt, file10.txt, file2.txt, ..., which would be the case with standard lexicographical sorting.

how many times specific letter repeated in python

To count how many times a specific letter appears in a string in Python, you can use the count() method of string objects. This method returns the number of non-overlapping occurrences of a substring in the string.

Here is a simple example:

# The string to search
my_string = "Hello, world!"

# The letter to count
letter = "l"

# Count the occurrences
count = my_string.count(letter)

print(f"The letter '{letter}' appears {count} times.")

In this example, the letter 'l' appears 3 times in the string "Hello, world!".

This method is case-sensitive, meaning it distinguishes between uppercase and lowercase letters. If you want to count a letter regardless of its case, you can convert the entire string to lowercase (or uppercase) before counting, like so:

# Convert string to lowercase and then count
count = my_string.lower().count(letter.lower())

print(f"The letter '{letter}' appears {count} times.")

This approach ensures that the search is case-insensitive.

how do i sort a dictionary by value in python

To sort a dictionary by its values in Python, you can use the sorted() function along with a lambda function as the key argument. The sorted() function returns a list of tuples, where each tuple is a key-value pair from the dictionary, sorted by the value.

Here's an example:

# Sample dictionary
my_dict = {'a': 3, 'b': 1, 'c': 2}

# Sort the dictionary by value
sorted_items = sorted(my_dict.items(), key=lambda x: x[1])

# Convert the sorted items back into a dictionary (optional)
sorted_dict = dict(sorted_items)

print(sorted_items)  # [('b', 1), ('c', 2), ('a', 3)]
print(sorted_dict)   # {'b': 1, 'c': 2, 'a': 3}

In this example:

my_dict.items() returns a view object that displays a list of dictionary's (key, value) tuple pairs.
sorted(..., key=lambda x: x[1]) sorts these tuples by their second element (the values), because lambda x: x[1] returns the value of each tuple.
If you need the result as a dictionary rather than a list of tuples, you can convert the sorted list back into a dictionary using dict(sorted_items). However, remember that dictionaries only maintain their order in Python 3.7 and above.

Sorting in Descending Order

If you want to sort the dictionary by values in descending order, you can add the reverse=True parameter to the sorted() function:

# Sort the dictionary by value in descending order
sorted_items_desc = sorted(my_dict.items(), key=lambda x: x[1], reverse=True)

# Convert the sorted items back into a dictionary (optional)
sorted_dict_desc = dict(sorted_items_desc)

print(sorted_items_desc)  # [('a', 3), ('c', 2), ('b', 1)]
print(sorted_dict_desc)   # {'a': 3, 'c': 2, 'b': 1}

This method allows for flexible sorting of dictionaries by values, whether in ascending or descending order.

count lines in string python

To count the number of lines in a string in Python, you can use the str.count() method to count the occurrences of the newline character , which signifies the end of a line. However, if the string ends with a newline character, this approach might not directly give the expected result since the last line does not need to end with a newline to be considered a line. Here's how you can accurately count lines:

my_string = """This is a string
spread across
several
lines."""

# Counting lines
# Adding 1 because the last line does not end with a newline character
line_count = my_string.count('\n') + 1

print(f"The string has {line_count} lines.")

This code counts the newline characters and adds one to account for the last line, assuming the string does not end with a newline. If the string might end with a newline (making the last character ), and you want to avoid counting an additional line in such cases, you could modify the approach slightly:

my_string = """This is a string
spread across
several
lines.
"""

# Adjust line count based on whether the string ends with a newline
line_count = my_string.count('\n') + (not my_string.endswith('\n'))

print(f"The string has {line_count} lines.")

This adjustment ensures that a trailing newline does not result in an extra line being counted, maintaining accuracy whether or not the string ends with .

sort dictionary by key python

Sorting a dictionary by its keys in Python can be achieved using the sorted() function, which will sort the keys, and then you can create a new dictionary with keys in the sorted order. Python dictionaries maintain their order as of Python 3.7 and later, so the sorted order will be preserved in the resulting dictionary.

Here's a straightforward way to sort a dictionary by its keys:

# Original dictionary
my_dict = {'banana': 3, 'apple': 1, 'cherry': 2}

# Sort the dictionary by key
sorted_dict = dict(sorted(my_dict.items()))

print(sorted_dict)

In this example:

my_dict.items() returns a view object displaying a list of the dictionary's key-value tuple pairs.
sorted() sorts these tuples primarily by the first element of each tuple (the keys), because that's the default behavior.
dict(sorted(...)) constructs a new dictionary from the sorted list of tuples, preserving the key order.

The resulting sorted_dict will have its keys sorted alphabetically: {'apple': 1, 'banana': 3, 'cherry': 2}.

Sorting in Descending Order

If you want to sort the dictionary by keys in descending order, you can use the reverse=True parameter of the sorted() function:

sorted_dict_desc = dict(sorted(my_dict.items(), reverse=True))

print(sorted_dict_desc)

This method keeps the dictionary sorted by its keys, in reverse order.

restrict the list with max item count in python

To restrict a list in Python to a maximum number of items, you can simply truncate the list to the desired length. This can be done using slicing. Slicing allows you to specify a start and end index, and Python will return a new list containing only the elements within that range. If you want to limit the list to a maximum number of items, you would start at the beginning of the list (index 0) and end at the index corresponding to the maximum number of items you want to keep.

Here’s how you can do it:

Truncating a List

# Original list
my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Maximum number of items to keep
max_items = 5

# Truncate the list
restricted_list = my_list[:max_items]

print(restricted_list)

In this example, restricted_list will contain only the first 5 items from my_list.

Ensuring the List Is Not Shorter Than the Maximum

The above method works well even if the original list is shorter than the max_items limit. Python slicing handles cases where the end index is greater than the length of the list gracefully by returning the entire list up to its actual length, without raising an error:

my_shorter_list = [1, 2, 3]

# Attempting to truncate to 5 items
restricted_list = my_shorter_list[:5]

print(restricted_list)

This will output [1, 2, 3] since my_shorter_list contains fewer than 5 items.

Modifying the List In-Place

If you need to modify the original list in place to restrict its length, you can use slicing on the left-hand side of an assignment:

# Original list
my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Maximum number of items to keep
max_items = 5

# Truncate the original list in place
my_list[:] = my_list[:max_items]

print(my_list)

Now, my_list itself is modified to contain only the first 5 items.

These techniques allow you to easily control the size of a list, ensuring it does not exceed a specified number of elements.

I have template.py in folders and subfolders. How do I change this filename to zemplate.py in all folders by using python

You can accomplish this task using Python by recursively traversing all folders and subfolders, then renaming the template.py files to zemplate.py. The os module in Python provides functions for interacting with the operating system, including file operations like renaming files. The os.walk() function can be used to recursively traverse directory trees.

Here's a Python script that demonstrates how to achieve this:

import os

def rename_template_files(root_dir):
    for root, dirs, files in os.walk(root_dir):
        for file in files:
            if file == 'template.py':
                old_path = os.path.join(root, file)
                new_path = os.path.join(root, 'zemplate.py')
                os.rename(old_path, new_path)
                print(f"Renamed: {old_path} -> {new_path}")

# Replace 'root_directory' with the root directory where you want to start renaming
root_directory = '/path/to/your/root/directory'
rename_template_files(root_directory)

In this script:

rename_template_files() is a recursive function that traverses all folders and subfolders under root_dir.
For each file found, if the filename is template.py, it renames it to zemplate.py using os.rename() function.
Replace 'root_directory' with the path to the directory where you want to start renaming the files.

Please make sure to test this script on a backup of your files or in a controlled environment to ensure it behaves as expected before running it on your actual files. Renaming files can be a destructive operation, so caution is advised.

100

PreviousPy Notes 3 NextTurtle Commands

Last updated 1 year ago

Importing the re Module

Basic Regex Functions

Compiling Regular Expressions

Examples

Flags

Special Characters and Sequences

Accepting Multiple Values

Accepting a List of Values

Accepting Zero or More Values

Accepting a Pair of Values

Using the glob Module

Using the os Module with Regular Expressions

Sorting in Descending Order

Sorting in Descending Order

Truncating a List

Ensuring the List Is Not Shorter Than the Maximum

Modifying the List In-Place

Importing the `re` Module

Using the `glob` Module

Using the `os` Module with Regular Expressions