I updated to Mojave. I remembered (again) that /etc/apache2/httpd.conf gets reset. This means that Apache may serve up local web pages, but local php sites may stop working. As always, I follow the instructions here to review how to get PHP working again. It’s an easy fix and PHP now runs again. What’s new? PHP 7 on the local machine as default. Hmm … I know I’m behind. It’s been a busy year.
I recently updated my laptop to run Python 3.7.0. I forgot that updates knocks out all the Python virtual environments that expect Python 3.6. I’ve worked my way through the virtual environments, removed them, recreated them and restarted them.
> rm -rf venv
> python3 -m venv venv
> source venv/bin/activate
I noticed that I don’t have a requirements.txt file to make it simple to reload Python modules. I’ve been creating that file as needed. I also noticed I was having trouble with loading the modules needed for an OpenCV project.
i installed OpenCV with homebrew and I knew it was outdated, so I upgraded it (and other dependent files) with brew. I reloaded all the related Python modules, but still could not get the sample file to run.
> brew outdated
> brew upgrade opencv
Instructions here show full steps for installing OpenCV. Note step 6 where a symlink is set up to point from the openCV libraries installed by home-brew and pointing back to the virtual environment
> cd venv/lib/python3.7/site-packages/
> ln -s /usr/local/opt/opencv3/lib/python3.7/site-packages/cv2.cpython-37m-darwin.so cv2.so
Note that file paths now show 3.7. If and when Python 3.8 or Python 4 show up, the paths will probably need to be updated to show the Python version.
I mentioned in another post about my plans for WorldWarIICasualtyProject.org. In short, the U.S. National Archives has scanned pages out of books that list American casualties that took place in World War II. I was curious to find out more, but discovered that those records did not exist in searchable form. I thought it would be interesting to figure out how to scan them (gif files, really!) and read the data as OCR.
I decided to go ahead on my initial plan: use a Python OCR module to read the scan. However … the Python module I tracked down (pytesseract) also required PIL (another module, part of the dependencies) and strongly suggested I install the python science packages. I figured I would need them at some point, so I installed numpy, scipy, matplotlib, scikit-image, scikit-learn, ipython, and pandas. ( https://www.learnopencv.com/install-opencv3-on-macos/)
At this point, I paused. I found several pages that suggested OpenCV be installed with Homebrew. That’s not a big deal because I use Homebrew for python 2/3. It gets confusing here. At one time, OpenCV was kept in a specialized area named “homebrew/science” but was moved to “homebrew/core”. I’m told “homebrew/science” is empty, so there should be no reason to link to it. We’ll see.
Note: use ‘> brew tap’ to list all taps connected for homebrew
Also note: opencv3 does not exist anymore. I think it has been renamed to opencv. Opencv2 has been renamed ‘opencv@2’. … So confusing …
Then there’s the question of linking OpenCV to “… Homebrew Python’s site-packages directory”. What? See https://www.learnopencv.com/install-opencv3-on-macos/
I’m sticking with these instructions: https://robferguson.org/blog/2017/10/06/how-to-install-opencv-and-python-using-homebrew-on-macos-sierra/ except for the part where I tap into homebrew/science (it doesn’t exist any more) and I install opencv3. (It’s been renamed to opencv).
I installed OpenCV through homebrew. Lots of dependencies were installed. Interestingly enough, I can see opencv through the default homebrew python3 install, but not in virtual environment I created for custom work. In other words:
>>> import cv2
However, when I go to the virtual environment set up for ww2cp, I don’t see it.
> source ww2venv/bin/activate
(ww2venv) > python3
>>> import cv2
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
ModuleNotFoundError: No module named ‘cv2’
So, following the instructions here: (https://robferguson.org/blog/2017/10/06/how-to-install-opencv-and-python-using-homebrew-on-macos-sierra/), I set up a symbolic link between homebrew’s openCV install and the site-packages inside the ww2 venv folder.
Now it works!
I run VirtualBox on my machine to test Ansible playbooks and to run (my legal copy of) Windows 7. I’ve received noticed from VirtualBox that there was a new version of VirtualBox available. (I run 5.1.28. The notice says 5.1.30.) After several weeks of delay, I decide to check it out.
I go to the VirtualBox website and discover that VB is now up to version 5.2.2. This is interesting. I wonder what they fixed? I like that it’s on minor version x.x.2, so they’ve tracked down the errors. I like it. I’ll download it and test it.
It looks good. It starts up as usual. The Windows 7 VM starts as expected. OK, let’s check out ansible. (Record scratch!!!) It won’t run any more. Vagrant 1.9.7 insists that it needs a copy of VirtualBox to run. I have VB installed. This can’t be right. I’ll force it and try again. Still not go with vagrant.
Let me run a Google search. It turns out that vagrant 2 won’t run VirtualBox 5.2 until the next minor update. What’s the newest version of vagrant available? 2.0.1. That’s it. I download it and attach the machine to a good internet pipe to handle the playbook.yml file. It starts up finally. A wasted afternoon on a machine that was not fixed until 9 pm.
As mentioned before, my web/database server infrastructure is becoming more complicated. I need to figure out a way to make things easier and repeatable for me. I’m going to test Ansible for server configuration. I also decided to try Capistrano for website deployment.
Originally, I was thinking of using Deployer. I liked the idea of having a PHP tool to deploy PHP websites, but then I realized I also want to deploy Node.js and Python WSDL sites. (By the way, deploying Python sites using Flask or Django as the web framework looks unusually complicated. More about that later.) For the PHP sites, I wrote a Phing script to collect everything I needed. My plan is for Capistrano to take that bundle and deploy that. We’ll see.
To use Capistrano, I need Ruby. I installed that somewhere when I took the Berkeley MOOC class.
ruby 2.1.5p211 (2014-11-13 revision 48405)
Wow, that seems old. I’ll update it. After some google searches, I settle on this set of instructions to upgrade ruby. Their process in a nutshell:
- install home-brew and git
- install rvm. Over rbenv? Uh, OK.
- install ruby
- install any needed gems
Homebrew are already covered. Git works as expected. They also recommend the latest version of the macOS, along with Xcode and the Xcode command line tools. I’ve got that covered, too. Let’s keep moving.
Next: install gpg? What is that? It checks cryptographic security of the rvm download. OK, sure. I installed the security key as described. (When I went through this process on a different machine, I forgot about the security key and did not have any issues with the rvm download.)
Next: download rvm. It looks simple enough. After the installation, I get a notice about two versions of rvm running on my machine. Apparently, I need to either source my .bash_process file or reload a terminal window, which does the same thing. OK. that’s working now.
Next: use rvm to install ruby. Am I seeing things? Why is it installing ruby into my home directory? Well,iIt seems to work. Ruby has the correct version. Final stuff: update bundler and nokigiri using the gem installer.
Everything looks good. Next, Capistrano.
I use the stock PHP version installed for macOS Sierra. In my case, it’s PHP 5.6.30. It runs well on my local web apps and it’s useful for my development environment.
I’m not going to stick with it forever. The vagrant environment in Laravel (Homestead) uses PHP 7. The vagrant machine I’ve seen recommended for development comes with PHP 7. I’ll move to it eventually.
During the spring, I was trying to clean up my brew environment to allow for a cleaner install. I paid attention to the warnings, but I accidentally ran “brew upgrade” without any modifiers. I ended up installing a ton of stuff that I don’t use regularly, including PHP 7. “brew services” shows that it’s not running, so that’s OK, but still … I was not paying attention and now I have PHP 7.
I use composer to keep all my PHP packages and modules updated and tracked. I added a new module to a composer.json file, so I needed to update it. I ran composer and I got the following message:
“Your requirements could not be resolved to an installable set of packages.
– This package requires php ^5.5.38 but your PHP version (7.0.15) does not satisfy that requirement.”
It turns out that “which php” points to /usr/local/bin/php, which is the home brew install. When I check $PATH, it turns out that /usr/local/bin appears before /usr/bin, where the stock install of PHP 5 is located. That’s why PHP 7 is the PHP appearing in the command line.
1. Change the $PATH order
No! This is a bad idea. By changing this, a lot of the home brew installs will use the wrong version. It seems too much effort to fix one item.
2. Update Apache to use PHP 7.
I have to do it anyway, just not now.
3. Change the php requirement in composer.json
I could change the requirement to “>=5.6.30” instead of “^5.5.38”. The ^ requires that PHP stay within PHP 5. ‘>=‘ only requires that the PHP version be greater than or equal to that number. It’s a band-aid, but it gets the updates running for tonight.
Recently, I upgraded my laptop to run macOS Sierra. Along with the OS updates, I also used brew to update python3. I was not paying attention and ended up upgrading PostgreSQL to the newest version, 9.6.2. I was unhappy that I was not paying attention, but I was glad that upgrades to PostgreSQL and MongoDB were also done with the upgrade to Python 3.
I use brew to start and stop PostgreSQL as needed, so I started it to check to see how it ran after the update. ‘psql’ gave me a version number, but I could not get it started to show me the schemas or tables from previous work. I also saw the following error message:
psql: could not connect to server: no such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/tmp/.s.PGSQL.5432”?
Very strange error message, so I started doing Google searches to see what could be wrong. There some dead ends involving checking for a PID file and checking permissions in /usr/local/var/postgres. Eventually, I ran into what looks like the answer in two related locations: a blog post describing upgrades to PostgreSQL and another one describing the upgrade process using brew. Apparently, I had two versions of PostgreSQL on my machine.
> brew info postgresql
confirmed it. Both pages described what I have to do, so I’ll do it. I should have known I needed to do this. Now I know.
As I mentioned earlier, I had trouble loading the free update to Windows 10. I now have a 5 GB file in my Windows 7 instance, but it never finished installing. It was stuck on 99% complete install for hours. I grew frustrated and decided to punt on the install.
Some websites mentioned that the Windows 10 update needed a fully patched and updates Windows 7 system in order to finish installing. It’s possible that may be true. It’s definitely true that I needed to patch Windows 7 as soon as possible. After installing an antivirus program (Avira) along with FireFox and Chrome, I decided to run the Windows update.
That did not go well either. I selected Windows Update from the Windows Control Panel and the “Other Programs” list on various occasions. I never got the impression that anything was happening. From other websites, I knew that the initial Windows update should take roughly 90-120 minutes. I would run Windows update 3 hours with no indication that anything was happening.
I decided to try to install Windows updates directly from the browser. My browser was immediately updated to IE 11. Immediately is relative. It still took roughly 90 minutes to download and install 14 updates for IE 11 only. Once it was done, I went back to update.microsoft.com to see if that would list files to download. I was told to set the update back to Windows Update. I did not know any better, so now going to update.microsoft.com bounces me over to Windows Update again.
I may try downloading Windows 7 SP1 by itself. Maybe that will take care of some of the missing updates. It’s possible my home download speed is too slow to download any information quickly enough. It would be nice to go somewhere with a big pipe and not a lot of traffic. Where could that be?
Microsoft announced that free upgrades to Windows 10 would end on July 29, 2016. I bought a full copy of Windows 7 to load onto a VirtualBox Virtual Machine several years ago, but never got around to installing it. I realized that this week would be a good time to get that project started and finished.
A while back, I created a ISO file of the Windows 7 installer DVD. I know that my current laptop does not have an optical drive, but the older one (the one with the failing trackpad) does. I suspect I bought the installer when I had the older laptop, but I also knew that I would need an ISO, if not for a new laptop, then for the desktop, which also does not come with an optical drive. (Weird.)
I created a new VirtualBox Virtual Machine to handle Windows 7. I remember I created a virtual optical drive in the virtual machine settings and attached the ISO to that virtual drive. Everything looks ready. Let’s start. Immediately, I get a message saying “Failed. No bootable medium found.”
I aborted and checked the settings again. There was a google search that suggested I hit the F12 key during boot up. That may have solved some issues, if I had tried it. I knew I needed an external optical drive any way, in case I sell or destroy the old laptop, so I drive down to the local Apple store and buy an overpriced optical drive.
I come back home and attach it to the new laptop. I can see the Windows 7 optical disk in the finder. I misunderstand how to load it in the Virtual Machine settings and end up starting the installer from the ISO. I know because if the installer were using the disk, I should have heard spinning sounds from the optical drive, which I did not.
It looks like Windows 7 is installing this time. I know it’s supposed to reboot to finish installation. What slips from my mind is that the second time, it’s supposed to reboot from the hard drive, not the ISO / optical drive. I keep interrupting the boot process by manually booting from the optical drive. It turns out that I installed several copies of Windows 7 into the virtual hard drive until I ran out of space.
I can see this is not working, so I start over with a fresh Virtual Machine. I decide not to add a virtual hard drive after realizing that the link to an optical drive already existed. When I start the blank windows machine this time, I was asked to pick a bootable drive. I was given the option of the drive with the Windows installer or the ISO (?). I pick the drive and go through the installation again, along with my extra second installation from the optical drive after reboot.
I read a bit online. (Google is your friend.) At this point, I realize that I need to let the reboot work without interference from me. I wipe out the Windows partitions created by the Windows installer and start over.
Finally, the Windows 7 installer works. I have a running Windows 7 instance running in VirtualBox. Next steps:
- upgrade to Windows 10 for free – nope, 99% freeze. See next post.
- load an anti-virus application – I’ve done research on free AV
- load all other windows updates – nope
- install Firefox and Chrome – eventually
All of that is for another time.
I always forget that major software upgrades in OS X reset permissions in /usr/local. When I checked my brew install, I saw that message. I played around with write permissions only, but in the end, did what brew recommended and reset the permissions as described. That allowed me to update brew and move on to the next task: updating Python3.
I was not as far back as I expected. I was sitting on Python 3.4. The upgrades went smoothly and I’m back to Python 3.5.1. I thought I had modules installed, but I’m not getting any list of local modules installed. It’s very possible, since I never did much with on the desktop. That will be the next task.