Python 3

November 30, 2021

Import Faker

Faker is a useful Python package that generates fake data. There's no need to include lots of external text files to generate random data.

#!/usr/local/bin/python3
from faker import Faker
from faker.providers import currency
fake = Faker()
fake.add_provider(currency)
Faker.seed()
for _ in range(10):
	print(fake.pricetag())

Sample Output

$7,604.87
$975.93
$54.21
$89,241.15
$91,565.93
$1,784.08
$6.09
$4,535.13
$22.87
$5.87

Need Same Results?

If you are writing Unit Tests, you may want to have the same list generated. To do this, simply add a value to the seed().

# Keep the same data output:
Faker.seed(10)
# Go random for each run
Faker.seed()

Random Words

Faker has the ability to add random words. In this example, the variable "randomwords" has a list of 20 words, I then shuffle that list and present the winning word of the day:

from faker import Faker
import random
faker = Faker()
randomwords = faker.words(20)
random.shuffle(randomwords)
print(f'Winning Word of the day: {randomwords[1]}')
print(randomwords)

Output:

Winning Word of the day: country
['seven', 'country', 'prove', 'husband', 'cause', 'wide', 'son', 'probably', 'small', 'treatment', 'property', 'policy', 'in', 'along', 'husband', 'cup', 'news', 'partner', 'wish', 'should']

Using Faker Profile

If you need just some of the basic elements of random data, you can use a profile or simpleprofile.

profile() - job, company, ssn, residence, current_location, blood_group, website, username, name, sex, address, mail, birthdate

simple_profile() - username, name, sex, address, mail, birthdate

Exmple Use:

from datetime import date
from datetime import datetime
from dateutil import relativedelta
def mydatecal(someday):
	caldate = relativedelta.relativedelta(datetime.now(),someday );
	myformat = "{} years old".format(caldate.years);
	return myformat
	
customer=fake.simple_profile();
print(customer['name'] + ' has the following email: ' + customer['mail'])
print("Birthday: " + customer['birthdate'].strftime("%B %d, %G"))
print( mydatecal(customer['birthdate']) + " today."+ "n")

Sample Output

Michael Ramos has the following email: russellamy@hotmail.com
Birthday: September 06, 1956
65 years old today.

Installing Faker

Use PIP to install Faker

pip install Faker

If you're not able to install using the pip command, use this:

python3 -m pip install faker --user
November 23, 2021

Working with Python Strings

Python Strings

Various code snippets on using strings in Python.

Getting a File to a String

This is useful when you have a list in a file and you want to use it in your code. At the end of this snippet, I shuffle the new list.

f = open('~/random/content/namelist.txt', 'r');
surnames = f.read();
f.close();
surname = surnames.splitlines()
random.shuffle(surname)

Replacing different words with a single replacement

An easy way to search/replace multiple items:

# Quick Utility Function
def replace_all(text, dic):
    for i, j in dic.items():
        text = text.replace(i, j)
    return text
# Sample String
story = "author was set to move to state"
# dictionary with mixed keys
my_dict = {'author': 'John', 'state': 'Massachusetts'}
newstory = replace_all(story, my_dict)
print(newstory)

Use the String as a List

What if I wanted the 3rd word in a particular string?

# Sample string
mytext = "Pleached Baloney Silurus Ramble Chemolytic Bidentate Malanders Brugnatellite"
# Use Split to break the string
newtext = mytext.split()
# Now display the third word, keeping in mind the counter starts at zero
print(newtext[2])

Counting the number of List Elements

thekeywords= ["sum", "fast", "snow", "Sawyer", "Pond"]
totallines = len(thekeywords)
print(totallines)

Looping through an List to output data

In this example, I am splitting up a string and shuffling the words:

import random
samplequote = "Today is gonna be the day that they're gonna throw it back to you"
samplequote2 = samplequote.lower().split()
random.shuffle(samplequote2)
# Create a function to display the Array, useful if you want to do something other than printing.
def print_list_elements(list):
    for element in list:
        print(element)
print_list_elements(samplequote2)
# Quick and Easy Way:
print("Really easy way:" + str(samplequote2))
# Show it as a new String, capitalize the first word in the sentence:
print(' '.join(samplequote2).capitalize() )
November 16, 2021

Ordinal not in range

Have you ever encountered an error where the "Ordinal not in range?"

Ordinal Not In Range

Sample Output:

UnicodeEncodeError: 'latin-1' codec can't encode characters in position 13821-13822: ordinal not in range(256)

According to various StackExachange comments, this error is thrown because Python can't encode some characters in a non Ascii string.

For me, this particular error occurred because I had bad data in my MySQL database - there was no codec checking done on INSERT. Apparently, PHP didn't have a problem handling the 'latin-1' codec - but using Python is now an issue.

Quick Easy Solution

My quick and easy solution is to have MySQL convert the text to ASCII. This is easy to do in-code:

cur.execute( "select date, CONVERT(teaserText USING ASCII), CONVERT(myTextData USING ASCII) from Order .... "")

Here's what happens: The MySQL CONVERT utility will convert the text in the teaserText and myTextData columns on the fly. The process is seemless and adds very little overhead while Python runs the query - your miles may vary if you have a complex database query.

Use Views

A better solution would be to create a view that would automatically convert the data, this way you can fix other columns if they cause issues:

CREATE OR REPLACE
ALGORITHM = UNDEFINED VIEW `order_vw` AS
select
    date_format(`w`.`orderdate`, '%M %e, %Y') AS `date`, 
   convert(`w`.`teaserText`
        using ascii) AS `teaserText`,
    convert(`w`.`myTextData`
        using ascii) AS `myTextData`,
    `c`.`name` AS `item`,
from
    ((`order` `w`)
where
    (`w`.`weblog` <= curdate())
order by
    `w`.`weblog`
November 9, 2021

View Python Exceptions in the Browser

To enable debugging your Python CGI, use the following in your Python header:

import cgitb
cgitb.enable()

Now when you run into an error, you will see the error message in your browser. Just be careful not to enable this for production pages. cgitb is part of the standard distribution package.

Official Description

If an uncaught exception occurs, a detailed, formatted report will be displayed. The report includes a traceback showing excerpts of the source code for each level, as well as the values of the arguments and local variables to currently running functions, to help you debug the problem. Optionally, you can save this information to a file instead of sending it to the browser.

Note: This doesn't handle all errors, for example, if you put in an illegal condition statement, you'll see a syntax error in the browser, but will need to run the python script from the command line to see the error.

Turn Off When In Production!

cgitb is a good thing to enable when developing code. Once the code is complete, remove the cgitb reference so that if customers encounter an error they don't see the code.

Python Name Error
Python Error page can show sensitive information!

Change the enable to display the data to a log file. Here's an example entry.

import cgitb
cgitb.enable(logdir=os.path.join(os.path.dirname(__file__), 'Weblog'),
             display=False,
             format='text',
             )

You can always setup a GitHub action to search for the "cgitb.enable()" before posting code into Production.

November 2, 2021

Formatted String Literals

One of the features I like in PHP is the heredoc string syntax because developers can include inline variables. This makes it easy to include variable within content.

Here's a sample snippet:

$content .= >>> end_here Hello, $world! end_here

In Python 3, you have the same functionality by using f-strings (otherwise known as Formatted String Literals). To enable this you simply prefix the string with an 'f'. For example:

content += f""" Hello {world}! """[1:-1]

or

print(f"Hello, {fname}, thank you for your input.")

Check out the Detailed Python Instructions on Formatted String Literals.

Practical Tricks with Python

Remove the First Line

If you get in the habit of starting text on the line immediately below the start of the heredoc, you should add a [1:-1] at the end of the heredoc. This is because in Python the heredoc actually starts after the third quote. This means that if you use heredoc to generate lists, the first item will be empty. Yikes!

I would recommend setting up a code snippet with the [1:-1] at the end so you don't forget.

In this example: I created a list of restaurants using a heredoc, converted it to a list, randomized the list order, and then displayed the results:

foodchains = """ Taco Bell Boston Market MacDonalds Sonic Fatburger Jack-In-The-Box """[1:-1] fastfood = foodchains.splitlines() random.shuffle(fastfood) print(f'Best place to eat is {fastfood[1]} and {fastfood[2]} then {fastfood[3]}')

Fix the Unicode Error

If you encounter the dreaded "UnicodeEncodeError" from a database import, you can easily fix it by using a simple convert trick within your string. You can apply a modifier to the Formatted String to convert the value before it gets outputted.

In this example the display text will be convert to ascii():

print (f'Name: {databaseName!a})

Another useful format command is '!s' which will convert the text to string.

Adding Commas to Digits

When you have a large number and want to "commify" the output, simply use this trick with formatting string literals:

totalSales = 20000
print(f'Formal: {totalSales:,.2f} and Simple: {totalSales:,}')
// Formal: 20,000.00 and Simple: 20,000

Execute Expressions

You can easily perform commands within the f-string. In this example, I am displaying the current date:

## Display Date as Month, ##, #### (Without the leading zero in days.)
from datetime import date
print(f'Today is: {date.today().strftime("%B %-d, %G")}')

## Another way:
import datetime
dt = datetime.datetime.now()
print(f'{dt:%A} {dt:%B} {dt.day}, {dt.year}')

Another example, making sure that the display name is properly capitalized.:

## Make Name Uppercase content += """ Welcome to the Forum {FirstName.capitalize()}! """[1:-1]

October 3, 2021

Unix Find Tips

Quick UNIX tip that I recently learned:

If your a UNIX power user, chances are you done search using the find command. The search may look like this:

find / -name python - print

The problem is that you may run into a lot of noise. In particular, you search may end up with a lot of these types of entries:


find: /usr/sbin/authserver: Permission denied
find: /Library/Application Support/Apple/ParentalControls/Users: Permission denied
find: /Library/Application Support/Apple/AssetCache/Data: Permission denied
find: /Library/Application Support/ApplePushService: Permission denied
find: /Library/Application Support/com.apple.TCC: Operation not permitted
find: /Library/OSAnalytics/Preferences/Library: Permission denied
find: /Library/Caches/com.apple.iconservices.store: Permission denied
find: /Library/Caches/com.apple.aned: Operation not permitted
find: /System/Library/DirectoryServices/DefaultLocalDB/Default: Permission denied
find: /System/Library/Templates/Data/private/var/audit: Permission denied
find: /System/Library/Templates/Data/private/var/root: Permission denied
find: /System/Library/Templates/Data/private/var/lib/postfix: Permission denied
find: /System/Library/Templates/Data/private/var/db/appinstalld: Permission denied
find: /System/Library/Templates/Data/private/var/db/diskimagesiod: Permission denied
find: /System/Library/Templates/Data/private/var/db/locationd: Permission denied
find: /System/Library/Templates/Data/private/var/db/analyticsd: Permission denied
find: /System/Library/Templates/Data/private/var/db/rmd: Permission denied
find: /System/Library/Templates/Data/private/var/db/knowledgegraphd: Permission denied
find: /System/Library/Templates/Data/private/var/db/nsurlsessiond: Permission denied
find: /System/Library/Templates/Data/private/var/db/fpsd: Permission denied

The problem is that you may lose matches in all the permission issues.

## Easy fix

There's actually an easy fix. Simply redirect the error messages. Here's all you need:

find / -name boston -print 2>/dev/null

Now when you search for something you only see the matches:


$> find / -name python  -print 2>/dev/null
/usr/bin/python
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Aliases/python
/usr/local/Cellar/ffmpeg/4.3.1_9/share/ffmpeg/python
/usr/local/Cellar/python@3.8/3.8.7/libexec/bin/python
/usr/local/Cellar/python@3.9/3.9.1_6/libexec/bin/python
/usr/share/file/magic/python
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
August 22, 2021

Running Python on the Command Line

Python3 Monitor

Have you ever run into a problem executing a python3 file where you would get the following error:

  • OSError: cannot open resource
  • env: python3: Operation not permitted
  • python3: bad interpreter: Operation not permitted.

Easy Fix

Changes are you have the following header in that Python file:

#!/usr/bin/env python3

To make it work, change the header to:

#!/usr/bin/env /usr/bin/python3

Reasoning

This is probably happening because Python3 is not pointing to the right version of Python3.

The easy fix, is put in the full path of Python3. For most installations, it should be /usr/bin/python3. (Apple does a good job of keeping up with the current Python version.)

File Permissions

One last thing...

If your still having problems, make sure that the file permissions are set correctly.

chmod 755 filename.py

November 7, 2020

Stack Overflow Trends

When you need help with something with computer programming, the best place to go online is Stack Overflow. You can usually find an answer to your question, or if it's new, someone can answer your question.

Over the years Stack Overflow has captured a lot of data from the questions that people have asked. They put together a trends section, where you can compare various things.

You can find out what Operating System is the most popular, what computer language is being used, what Javascript framework people are using.

Website Description

See how technologies have trended over time based on use of their tags since 2008, when Stack Overflow was founded. Enter up to 15 tags to compare growth and decline.

Stack Over Flow Trends
https://insights.stackoverflow.com/trends

You can use the links on the page for popular searches, or type in tags that you want to compare against.

For "fun" see if you can guess what caused certain things to trend lower.

July 19, 2020

Why You Should Learn Tech Skills

With the evolution of technology, there's an imminent change in every industry and they will all become tech companies in the future. The need for using tech tools and hiring tech talent is increasing every day, and it's the only way for workers to stay ahead.

Should we all learn at least one tech skill? Yes, we should. Regardless of whether you want to work in a tech company or not, having a tech skill at your disposal is something that will level up your experience and make you more prepared for a professional career.

Despite what you may be thinking right now, you don't necessarily need to have a bachelor's degree in computer science to learn a tech skill. There are cheaper, short-term options that you can try like coding bootcamps or online courses. Here, we'll show you some of the reasons why you should start learning a tech skill and where you can learn them.

It Makes You a Problem Solver

Most tech skills, like data science, software development, or machine learning, function to solve a problem in a logical sense. Learning how to code will make you practice logic every day, see problems at a bigger scale, and discovering new paths to solve problems.

This isn't something that will only help you pay bills, learning how to code provides you deep analytical thinking and you'll soon become a problem solver once you start practicing every day. Problem-solving skills can be applied to every situation, not only when in coding.

It Makes You Future Proof

Almost every company is becoming tech-focused regardless of the industry. Just think about how many jobs that used to be manned by humans that are now automated by machines.

Almost everything is automated these days, from self-driving cars that will replace drivers to bookkeeping software that doesn't need human intervention. This leans the scale for people who have programming or any other tech skill. Being able to understand these activities will help you to ensure your role in the future, where almost everything is automated.

Most Tech Jobs Are Lucrative

Have you ever browsed Glassdoor to see the average salary of a data analyst, machine-learning engineer, or even a software developer? Most of these professions earn from $77,000 to over $123,000.

Salary rates differ in every case, but let's take data science as an example. Data analysts make from $100,000 to $123,000. The justification for this is there's a huge demand for them but supplies are low. As a result, data scientists end up having great salaries. This phenomenon occurs in several other tech jobs.

One should also consider the case of a software developer, who, on the other hand, is highly demanded yet supplies are steady. Software developers make less than data scientists, but the annual salary is still approximately $100,000 per year.

Tech Skills Help Companies Save

Saving money is important in everyday life. This is also true for companies that wish to expand. Having a basic knowledge of computer science will help you not only save money but also save time.

Calling the tech support department can be frustrating as the solutions it provides may not be suitable. Time is a resource that can be wasted when dealing with unnecessary problems you can solve yourself.

And the same happens with projects. If you have an idea and want to bring it to reality, but no one understands your explanation, the project might not be accomplished. Having the tech skills required for that specific project will help you generate desired outcomes.

Most Wanted Tech Skill

Most Wanted Tech Skills

Now that you are familiar with some of the tech skills, it is time to consider which path you want to take: a bachelor's degree or a tech Bootcamp. Regardless, you should be able to learn something you enjoy with a good salary. Here are some of the most in-demand tech skills these days:

Software Development

The ability to develop software gives you the flexibility to choose which tasks to automate at your company. We know some tasks can be repetitive and time-consuming. Processes like verifying data, dealing with accounting math, and doing statistics are tasks that a computer could do, and perhaps even better than us.

Besides, if you're creating your own company, and want to build an in-house digital platform, software development skills will help you a lot. There's a school where you can learn this skill in 20 weeks. Thinkful's immersive software development course will prepare you for the development of killer apps. You'll learn all fundamentals of software engineering and programming languages like Node.js, MySQL, and others.

Data Science

If you enjoy math and statistics, data science is for you. Data science refers to the ability to find meaningful insights in large datasets to make better decisions, prove a hypothesis, or find patterns. To become a data scientist, you need to understand programming languages like Python.

Galvanize is one of the leading schools in data science. Galvanize Data Science Bootcamp is oriented towards decision making. You'll learn how to work with machine learning, the statistical process, data analysis, and data visualization.

UX and UI Design

'Design' sounds like something artistic, right? Yes, but UX and UI design are far the traditional sense of 'design'. UX and UI design are related to user experience and how to improve it.

However, UI is more oriented to the interface and functionality of a software product. There's a reason why many users prefer to use Apple's iOS. The operating system is intuitive and comfortable for users. This is what UI design is capable of.

With Springboard's UX and UI design courses, you won't need prior skills or experience to learn. You'll learn everything you need to know about this skill from scratch and you'll be trained with practical classes with client-facing practices.

Mobile Application Development

We're all mobile users and we understand the importance of mobile apps. The reason why mobile app developers are in high demand is every company wants to have an in-house mobile application for users. These days, we spend more time on our phones than any other device.

Mobile developers work with Android and iOS operative systems, with each one working with a different language. Android developers work with Java, while iOS ones work with Swift or Objective-C.

App Academy's mobile development course will guide you through all of the processes of creating a functional mobile app. You'll have real-time practices that will allow you to track the performance of your project.

In Conclusion

No matter which path you take, learning a tech skill will open many opportunities that will boost your career or company. We know that technology is always changing, therefore, you need to stay ahead.

July 10, 2020

iTerm2 Shortcuts

iTerm2 is a very popular replacement application for Apple's Terminal. The application makes using terminal services a lot more productive. One great tool is the Key Mapping.

i Term Screen Shot2020

iTerm2 KeyMapping

If your copying/pasting the same commands every time you open up iTerm2, you should know about the Key Mapping functionality. You can easily set up keyboard short cuts to perform some common commands.

Examples

  • ps -ef | grep chrome ## Show active processes
  • tar -cvf- . | gzip -9 > files.tar.gz ## Tar and Compress the current directory
  • bbedit /etc/hosts ## Change the local host file
  • su - apachectl restart ## Restart apache
  • mysqldump website >mysql-backup.sql # Quickly Backup the mysql database
  • sudo -u company /home/company/.virtualenvs/dev-ppc-two/bin/python run_report.py

i Term2 Steps Keys
Follow the steps below to put your own KeyMapping

Setting up the KeyMapping

It's really easy to set up KeyMapping. You do it for each profile. Which makes it great as you may have different commands for each server that you are connecting to.

  1. Open iTerm2
  2. Type Command , to open up Preferences
  3. Select the Profiles on the Top
  4. Select a Profile Name in the left panel
  5. Then Select the ‘Keys' sub menu
  6. Click the ‘+' under the row of "Key Mappings"
  7. Click on the "Click to Set" and define your hot key
  8. Under the Action Menu scroll and search for "Send Text"

As you can see there are a lot of action items. iTerm has a nice document of all the capabilities that you can do. Some tricks you can do:

  • Quickly open up a profile that is tailing the error log file using the "Split Horizontally with Profile."
  • Switch between live and dev sites using the "Change Profile" action.
  • Search for ‘Errors' and ‘Warnings' that you may have missed using the "Find Regular Expression.."
  • Check out various AWS instances by opening the profile in a new tab via "New Tab with Profile."