I run several pfSense boxes throughout my network. Although the platform doesn't have an API, and it can be a pain to configure manually in certain cases, it's generally very reliable once running, and because it's essentially a skinned BSD, it's very easy on resources. There's also a really nice self-update feature that I use to get things to the newest release when they're available.
It's that last feature that bit me in my butt Sunday night. After doing the upgrade at midnight or so, I went to bed after everything seemed to work alright, but then this morning, I started getting reports that people couldn't log into the captive portal that we use for our "guest" wired connections.
I thought, "That's strange...everything seemed to work after the upgrade, but I'll check it out", and sure enough, as far as I could tell, all of the networks were working fine on that machine, but there was no one logged into the captive portal.
Taking a look at the logs, I found this error:
logportalauth: Zone: cpzone - Error during table cpzone creation.
Error message: file is encrypted or is not a database
Well, hrm. "Error during table cpzone creation" is strange, but "file is encrypted or is not a database" is even weirder. Doing a quick google search, I came across this thread on the pfSense forums where someone else (maybe the only other person?) has encountered the same problem I have.
The thread on the forums suggests to shut off the captive portal service, remove the .db files, and then restart the service. I tried that, and it didn't work for me, so what I did after that was to shut down the captive portal (to release any file locks), remove the db files, and then from the text-mode administrative menu, force an re-installation of pfSense itself.
Although I haven't actually tested the captive portal yet (I'm at home doing this remotely, because #YOLO), a new database file has been created (/var/db/captiveportalcpzone.db) and inspecting it seems to show sqlite3 working:
[2.2-RELEASE][root@host]/var/db: sqlite3 captiveportalcpzone.db
SQLite version 184.108.40.206 2014-11-18 20:57:56
Enter ".help" for usage hints.
seq name file
--- --------------- ----------------------------------------------------------
0 main /var/db/captiveportalcpzone.db
This is as opposed to some of the older database files created prior to upgrade:
[2.2-RELEASE][root@host]/var/db/backup: sqlite3 captiveportalinterface.db
SQLite version 220.127.116.11 2014-11-18 20:57:56
Enter ".help" for usage hints.
Error: file is encrypted or is not a database
What I don't understand is that the normal way to convert from sqlite2 to sqlite3 is to dump and restore, but it doesn't look like this process did that at all. It would be incredibly easy to do a database dump/restore during an upgrade, ESPECIALLY when revving major database versions like this.
Anyway, this kind of experience is very unusual for me with pfSense. Normally it's "set it and forget it". Hopefully this will work and I can get back to complaining about a lack of API.
I've got a problem. Our school is expanding, and we're constantly hiring people. We're hiring so many people that they won't actually fit in the building we're in. Because of that, we're having to expand outside of the building we've been in for years. Part of that expansion is extending my networks across campus (and in some cases, farther).
The network that I run is really old. Like, it actually predates the network at the central university. I've got around 50 VLANs, and now that we're growing outside of this physical environment, I've got to extend those layer 2 broadcast domains to the other buildings. I have a good relationship with the central network folks, and although most of my VLAN IDs collide with theirs, they assigned us some IDs that we can use on their infrastructure. Now, I just have to translate my VLAN IDs to their VLAN IDs.
My network core is a pair of Cisco Nexus 5548s. When I was planning this migration, I didn't worry at all, because the documentation clearly declared that the switchport vlan mapping command was supported. The only weird thing was, when I went to set up the VLAN translation, the command wasn't found. It was in the docs, but not in the CLI. Weird, right?
So I did what you do when you pay ungodly amounts of money for Cisco support: I opened a ticket with the TAC.
I had been operating under the assumption that my device would be able to perform VLAN ID mapping on an interface, but I can't figure out how to do it.
Is it possible to map VLAN IDs across a link? I have a trunk to my provider across which I need to send several vlans, but my IDs collide with those in use there. I was hoping to use the equivalent of "switchport vlan mapping", but it doesn't appear to be in my release.
Can you please advise me?
I got back what may be the best response from tech support ever. Emphasis my own:
My name is XXXXXXX and I will be assisting you with the Service Request 633401489. I am sending this e-mail as an initial point of contact and so that you can contact me if you need to.
As I have understood it, "switchport vlan mapping" command does not exist in 5548
Allows for the merging of separate Layer 2 domains that might reside in a two data centers that are connected through some form of Data Center Interconnect (DCI).
So I can understand why you were under the impression that this platform supports this feature however I must state that the document is incorrect here.
I have verified with the Technical Marketing Engineers and it has been confirmed that there are no plans to support vlan mapping / translation on Nexus5500 platforms however as of today; Nexus5672, Nexus 6000 and Nexus7000 do support this feature in 7.x release.
Please let me know if there is anything else I may assist you with .
So that was, you know, less than helpful. And I still need to get those VLANs over there. How are we going to do this?
For now, I'm doing it the old fashioned way. Crossover cables.
Normally, when you move VLAN traffic around, you use a dot1q trunk. Each layer 2 frame gets a header when it leaves a switch that tells the remote device (usually a switch) what VLAN the packet belongs to. So, VLAN ID 10 gets a header that says "this frame goes to VLAN ID 10", which allows traffic from VLAN 10 and VLAN 20 to be sent over the same physical link and still be kept separate.
Since the VLAN ID is encoded in the frame, it'll cause problems if the VLAN ID I'm using means something else to the other network. But, since the only thing the other end cares about is the VLAN ID, if I can send my traffic over to the other network on the proper VLAN ID, then they're happy. To do that, I need to bridge the networks. The easiest way I know how to do that is to take an access port on VLAN A, and an access port on VLAN B, and plug a single cable into both of them (after disabling spanning tree, of course). Yes, this sounds insane. Yes, it might actually be insane. But this is how I did it, and it worked the first time.
The bad part is that I'm currently burning two physical ports for every VLAN I need to translate, and this isn't tenable over the long-run. Fortunately, the Juniper switches on the remote side of the network link support translation, so I believe that we should be able to do it the "right way". The sooner the better, because I feel dirty.
Ron Popeil introduced "Set it and forget it" into the lexicon, and really often, we sysadmins take that to heart when it comes to services and software. Blessed are those installations that reliably and quietly update on their own in the background, for they lead to full nights of sleep. But unfortunately, those kinds of things are few and far between.
I get to be part of a team now, which means that the Windows admin is responsible for Patch Tuesday shenanigans, and our Linux admin makes sure that apt-get auto-upgrades the important packages. Since I run the VMware infrastructure, I get to make sure that it is up to date with security patches and so on.
To help with this process, VMware provides the vSphere Update Manager. It runs in vCenter (so you do need to be using a licensed version), but it allows you to apply policies to VMs and hosts, and then scan them to make sure that they comply.
I have known that it existed for a while, but I hadn't spent any time learning it. I had done some upgrades on VM hosts from 4.1 to 5.0, and then from 5.0 to 5.5, but I was doing it manually, by upgrading from the ISO via remote console. That's kind of a drag, and it's completely manual, and just the ugliness of it made me itch to the point where I finally decided, "Alright, self, we're going to check out Update Manager and figure out how to make it work."
I was only hesitant because I'd heard from a few people who had bad experiences, and the "word on the street" from those people was that they found it unreliable and had stopped using it. Maybe in the past, it has been, but as of right now, I kind of consider it a god-send, because it sped up the host-upgrade process by a lot.
I'm not going to tell you how to install it or even really how to use it. There's much better documentation from VMware on that, but if you want a quick howto, I'd recommend checking out this video on YouTube from SysAdminTutorials:
Technically, it covers vSphere 5.1, but the 5.5 is very similar. Note that VMware has been encouraging us to use the web client for everything, but UpdateManager still requires the installed DotNet client to wrangle updates. At first, I absolutely loathed the web client, but after using it for a year or so, I only strongly despise it.
If you check out Bob Planker's 9 Things You'll Love about vSphere 6.0, you can see that some things will be improved, but alas, Update Manager will still live in the installed client. Chris Wahl writes that performance charts are available and usable in less than half the time. Which means that you can probably load them before you either get bored or forget why you visited that page in the first place. Seriously, the web client is bad, but there are some features that make it worthwhile. Update Manager just isn't one of them.
So anyway, at this point, all of my 15-ish vSphere hosts are up to date on the current release of vSphere Hypervisor aka ESXi, and I'm very happy that I spent the hour or so I needed to become comfortable with update manager. If you aren't using it, I'd suggest you take a look, too. It's not as bad as you may have heard!
I've been taking a bit of a break from the whole "social" scene, both on twitter and here on my blog. I've been busy, I haven't felt like writing, and somewhere in the middle of all that, Boston took amazing amounts of snow, so I've been unburying my car again and again. It's been a busy couple of months, but after not being active for so long, it feels good to fire up the blog editor again.
So what has happened since I wrote last? Well, some re-organization here at work. My boss, the esteemed David Blank-Edelman has left to be the "Technical Evangelist" for Apcera, an enterprise platform provider. I've enrolled in classes here at Northeastern, since I get free tuition, and in terms of IT stuff, I've been spending some quality time with my VMware infrastructure. If you follow me on Twitter, I flooded your stream with Virtualization Field Day 4 info, and I'm in the middle of evaluating a couple of the virtualization management solutions from that. In particular, I'm having a great time with VMTurbo, so expect something informative on that, and I'll be getting to play with some other fun stuff shortly.
Basically, I want to get back into blogging like I used to. Not every entry is going to be amazing, but I'm hoping that I can start to contribute again, in some way that will provide something to someone. We'll see how it goes. Thanks for reading.
Happy New Year, and welcome to 2015. This new year will feature an array of exciting things, like writing the wrong date on everything for the better part of a month, and still trying to figure out what to call the decade that we’re currently halfway through.
One of the first things I’m going to be doing this year is to attend Virtualization Field Day 4, in beautiful (and more importantly, warm) Austin, TX. I’m really excited, not least of which is because the average temperature in January is about 30 degrees above my home in Boston. But I’m also really excited to be a part of an outstanding team of individuals brought together to meet companies doing exciting things in the virtualization space.
3 x vExpert, blogger @ mwpreston.net and a typical Canadian eh!
There are a lot of the folks that I know from former events, but there are several I haven’t had the pleasure of meeting yet, and I’m really looking forward to it. Because Tech Field Day delegates help vote on future attendees, you don’t wind up with people there who aren’t interesting and knowledgeable in some way. Somehow, I slipped by, but given my almost encyclopedic knowledge of Star Trek, I fit right in, so they let me keep coming back.
The other side of the Tech Field Day coin is the people we’re brought together to meet with - the vendors! If nothing else, my experiences with Tech Field Day have taught me that not all vendors are bad. A lot of them really get technology, and they care about implementing it well and treat their customers as partners instead of just sources of income. Those companies are a pleasure to talk with, and they tend to have the best presentations at TFD events.
This will be the fourth Virtualization Field Day event (way back in the day, I actually ran the VFD2 event when Stephen couldn’t make it! No pressure!), and Stephen has rounded up a really interesting collection of companies in the virtualization space:
There are several companies there that I suspect most people are familiar with, but some of them are new to me, and I’m going to be going back and doing my homework. As I get my research done, I’ll be posting blog entries so that you can get some background, too. I’ve found this kind of thing to be great prep work for TFD events. Some of the TFD attendees are in the solutions space, and it’s their job to stay current on everyone in the market. I don’t have that kind of time, so I’ve got to play pickup. I figure most of you are in the same boat as me, so we’ll learn together!
Anyway, thanks to Stephen and Tom for having me back. I always enjoy my time with the TFD delegates, and I’m certain that this event won’t be any different. The event starts January 14th, so make sure to watch this blog for more updates!
Hi! I'm Matt Simmons, author of the Standalone Sysadmin blog. I've been a small infrastructure administrator since aound 2001, nearly always on my own, hence the "Standalone Sysadmin" title.
This year, I started working at Northeastern University's College of Computer and Information Science, so I'm getting a taste of a larger environment,but it's still small in the grand scheme of things, but I'll be passing along this new experience as I go through it!
I started this blog in May of 2008 and have thoroughly enjoyed every interaction I've had because of it. Thank you for visiting!