Dell FlexAddress with Redundant CMCs “Not Working”

Problem Description

With Dell’s FlexAddress technology enabled on a Dell PE M1000e, after changing a HBA or network card that has a different WWN/MAC than the original (it should, of course!), the FlexAddress WWN/MAC assignment does not take effect, and the mezzanine card’s OEM WWN/MAC prevails.

Hardware Topology

  • Dell PowerEdge M1000e chassis
  • Redundant CMC (chassis management controller) modules
  • single Dell FlexAddress SD card installed in one of the CMC modules
  • The other CMC (redundant) module does not contain a FlexAddress SD card
  • Dell Blade (mine are M600-series half-height)
  • Fabric A enabled (for onboard networking), or one or more mezzanine cards installed for Fabric B or Fabric C

Solution

One or more of the following steps are applicable:

  1. Ensure the CMC that contains the Dell FlexAddress SD card is active (not in standby). See page 179 of the “Dell PowerEdge M1000e Chassis Management Controller Firmware Version 5.0 User’s Guide“. If it is in standby, a graceful failover can be performed by invoking the cmcchangeover command. This will only affect out-of-band management (OOBM), and not the production system.
  2. Perform a virtual reseat (not restart) of the blade module, e.g. for blade in slot 10 racadm serveraction -m server-10 reseat -f . This is a “known” issue in the context of upgrading mezzanine card firmware, see Flex Address doesn’t work after Broadcom FW Update. This assumes the specific blade is down for maintenance anyway.
  3. Ensure the Dell FlexAddress SD card is not locked. With the angled corner to the top right, the sliding lever should be in the “forward” or “up” position. The FlexAddress card is installed in the SD card slot on the underside of the CMC board; the CMC module should be completely removed to inspect this. This procedure also only affects OOBM. (Hint: stick around to hear the fans spin up after re-inserting the CMC module!)
  4. Ensure that FlexAddress is enabled for the specific chassis slot. This can be done within the CMC web interface GUI, or telnet/SSH/serial CLI. Enabling/Disabling FlexAddress can only be performed when a blade module is installed in the slot, with iDRAC initialized (I think?), and the state of the blade is powered off. Changing the state of FlexAddress for a powered-on blade is not possible.
  5. Alternatively, using two Dell FlexAddress SD Cards – one in each CMC – will avoid guessing which CMC has it installed.

 

MongoDB: Migrate and Merge All Chunks in Shard

Today I was working in a MongoDB 3.0 sharded cluster environment. There was a particular sharded collection that had 300-some-odd chunks evenly distributed within each shard (thanks to the balancer). These chunks happened to be empty, and in need of some pre-splitting for near-future use.

I ended up writing MongoDB shell scripts to handle the migration of all the chunks to the primary shard, and to merge all of the chunks to a single chunk. The scripts adhere to the following:

  • Authenticate a clusterAdmin user against the admin database. (I actually used a “root” role user.)
  • Read the config database for sharding topology and chunk distribution.
  • Any “write”-like commands use sharding helpers where possible, and runCommand otherwise. No “write”-like commands use CRUD operations on the config database.

Don’t forget to stop the balancer before running these scripts, and then start the balancer when they’re done.

Step 1: Migrate Chunks to Primary Shard

It is impossible to merge chunks that are not on the same shard. First, it is necessary to migrate all the chunks, and for the sake of some standard, on the primary shard.

Run the script as follows:

Remember, authentication happens in the script, so no need to pass -u or -p credentials via the CLI. The script automatically finds the admin and config databases as well, so connecting to the admin database is not required.

Step 2: Merge All Chunks

Now that all the chunks are on the primary shard, it is time to merge them into a single chunk. Only contiguous chunks can be merged, and only two chunks at a time can be considered. If there are hundreds or thousands of these 64MB chunks, this can be tedious if done manually. The following MongoDB shell script finds a pair of contiguous chunks and merges them; it repeats this process until there are no more chunks left to merge.

Just like with the previous migrate script, run this one the same way:

Further Considerations

This exercise is also beneficial as prep work for:

  • Unsharding a collection
  • Unsharding a database
  • Changing the shard key of a sharded collection
  • Manually re/pre-splitting of shard chunks

MongoDB 3.0: mongos CentOS 6 service scripts

Premise

When installing mongos (not mongod) on CentOS 6 using the RPMs provided in MongoDB’s repository, no startup/service/init.d scripts are created, or really anything for that matter, to aid in managing mongos as a service. All the RPM provides is the binary.

If you want to treat mongos as a service, there are quite a few steps to follow beyond just setting up a script in init.d:

  • mongos YAML configuration file
  • mongos sysconfig service overrides
  • SELinux port definition
  • a user to run mongos

Most of the time there are many mongos processes distributed within a MongoDB cluster. Configuring each of these can be a hassle, let alone installing and managing them, unless you use an automation tool.

Be sure to read “Install MongoDB on Red Hat Enterprise or CentOS Linux” first.

mongos Service Scripts

Below are the 3 files I use when preparing a mongos cluster:

  1. The init.d service script for managing the mongos process
  2. A base mongos configuration file in YAML
  3. An automation script to prepare the CentOS 6 environment for mongos

/etc/init.d/mongos

The following init.d script has been retooled from the stock MongoDB 3.0 mongod for mongos. Watch out for the subtle differences.

/etc/mongos.conf Base YAML Configuration

Be sure to set sharding.configDB  appropriately.

Post-Install Environment Setup

Provided mongodb-org-mongos has already been installed, the following script will do most of the legwork to get the CentOS 6 environment up and running. Note: The script is pulling the configuration file and service file dependencies from a web server using wget; this should be tailored. It performs the following:

  • Add a mongod user (yes, with a d) with the same settings that would be supplied from mongodb-org-server.
  • Add a placeholder file for sysconfig.
  • Add the mongos.conf file with the base configuration noted in previous (wget).
  • Add the init.d service script (wget).
  • If SELinux is enabled, set the contexts for the files created in previous, and add the port definition via semanage. Note that it will automatically install the required tools to perform this task if required.
  • Add and enable the new mongos service

SELinux and MongoDB

The automation script above does not add all of the required SELinux policies required for mongos.  Install the mongodb-org package to initialize all of the policies; it can be removed later if need be while keeping these policies intact. Finding the script used by MongoDB (or making one up) is a task for another day. A policy dump from semanage is as follows:

 

RegExp: Matching Balanced Parenthesis and Quotes (greedy, non-recursive)

Solution

I need to match all the text within balanced parens, single quotes, or double quotes, but don’t snag on the content within, i.e. be greedy! “Balanced” means there should be a corresponding single or double quote, but not mix them, or there should be a closing paren for each opening paren.

PCRE:

The match will be in capturing group #3.

In short, this regular expression uses a positive lookahead (?=...)  within the lookaround conditional  (?...)  and the backreference \2  to match the initiating character. If it is a single quote or double quote, it was contained within \2 , but if it was an open paren ( , it is not contained in a capture group and therefore it should match a close paren. In retrospect, this could be enhanced, and also needs some tweaks for JavaScript compatibility since lookaround conditionals are not supported.

Do note that this matches the beginning and ending of the string ^$ , so may require tweaking for other applications.

Test Cases

regex101 link: https://regex101.com/r/fG4rZ8/3

Matches:

These will match hello , hello world , or in some cases 'hello world  for the last test case.

Doesn’t Match:

Node.js: Break on Uncaught Exceptions

In the event the implemented error handling isn’t what it’s all cracked up to be, having the Node.js debugger break on uncaught exceptions is quite simple:

Assume the following code, where bogus  is undefined:

Line 3 will terminate the application (normally), and log an exception in stdout (or the console, a log, etc.) — even when Node.js is in debug mode.

To debug this uncaught exception JIT, the implementation may be as follows:

Run the above application (whose filename is “index.js”) as node --debug index.js  and after 2 seconds (notice the timeout) the exception will be uncaught, but the process will break on line 5 for debugging.

See this gist for an auxiliary and shortened reference.

Migrating Kickstart from CentOS 6 to CentOS 7 + ESXi VMware Tools

In another post I described how to install CentOS 6 via a kickstart file (be sure to check out the “Kickstart Sample Configuration” section). CentOS 7 was recently released, and with that I needed to also use a kickstart configuration. However, simply using the previous kickstart configuration was not as easy as copy-and-paste (besides updating the release version in the repository configuration).

Summary of Kickstart Changes

There were a few changes that needed to be made for a base/core installation of CentOS 7:

  • Include  eula --agreed (read the documentation)
  • Include  services --enabled=NetworkManager,sshd (read the documentation)
  • Update the install packages list ( %packages  section)
  • CentOS 7 is also a bit more strict with the kickstart file, so I had to explicitly include %end  where applicable
  • CentOS 7’s default file system is now xfs, in CentOS 6 it was ext4, so consider updating the automatic partitioning to use xfs
  • Package groups @scalable-file-systems, @server-platform, @server-policy, and @system-admin-tools no longer exist – I haven’t located suitable replacements yet
  • Things like ifconfig are no longer included by default (they are now deprecated), so if you need them be sure to include net-tools. You should be using ip by now anyway.

Kickstart Sample Configuration

And now, an updated kickstart config for CentOS 7, with consideration of the previously mentioned updates (compare it with the previous Kickstart Sample Configuration mentioned in the other post). I also chose to include some extra packages that don’t exist by default with a @core  installation.

 

VMware Tools Change

If you’re like me and use ESXi, I’m currently on version 5.5 and 5.5u1, the installable tools for integrating with ESXi is a nice treat. However, the repository location has changed specifically for RHEL7, and so have the packages.

Add VMware Tools to YUM

Put the following repo configuration in /etc/yum.repos.d/vmware-tools.repo:

Then update yum and install the tools:

 

 

Node.js: Dynamic Event Listener Binding and Removal

Read this SO question first: Removing event listener which was added with bind

Now that we’re on the same page. I’m going to make this short’n’sweet by providing a premise and letting the code do the talking: The example provided in the SO post was all fine and dandy, but was for the case of single-use. I needed a way to clean up event listeners on a Node.js net.Server and associated socket connections.

 

 

Internet Calendaring (iCalendar): Repeating Events dot ics

Premise

So Sysadmin Day is on the last Friday in July every year. I wanted to add this to a Google Calendar (GUI), but for the year 2014 (July 25) it would always mark it as the fourth Friday. This was problematic since July 2015 has five (5) Fridays. By the way, it worked just fine if I started the event in July 2015.

After some initial searching, I ran across a few resources with hints, but these were focused on the last (week)day of the year:

I decided to roll up my sleeves and give it a whirl in ics format. I started to read up on RFC 2445 “Internet Calendaring and Scheduling Core Object Specification (iCalendar)” over at IETF. After realizing it was 148 pages long I decided to create a working event in the calendar, export it, extract and modify the event, then import the event back into the calendar. This worked.

Source & How-To

And now the ICS source for System Administrator Appreciate Day:

Source Explanation

  • BEGIN/END VCALENDAR: Entire calendar encapsulation. (Hmm, I wonder if multiple calendars will work? Need to read the spec.)
  • BEGIN/END VEVENT: Single event encapsulation within the specific calendar.
  • DTSTART: The start date of the event; this is an all-day event so there is no time nor timezone.
  • DTEND: Like  DTSTART, only for when the event will end – this is the next day for an all-day event.
  • RRULE: This is the bread’n’butter. The frequency is monthly every 12 months (so once per year) on the last (-1) Friday (FR);   1FR would be the first Friday,   2FR would be the second,   -2FR would the second-to-last Friday, etc. See §4.3.10 “Recurrence Rule” on page 40 of RFC 2445 to start.
  • SUMMARY: The formal name (title) of the event.

Make It & Import It

  1. Create a new plain text file (after all, ICS/iCalendar format is just text/calendar!)
  2. Copy and paste from   BEGIN:VCALENDAR to   END:VCALENDAR.
  3. Save it as whatever you want, in this case I’ll call it   sysadminday.ics.
  4. Import to an existing Google Calendar.
  5. ????
  6. EAT:CAKE!!!!

Basic PHP Class Serialization

Just a quick brain dump regarding serializing classes in PHP. There is nothing advanced about this, and TBH I haven’t even read up on the caveats of serializing classes in PHP – perhaps a task for another day.

Judging from the string output after serializing the class, it appears that it is simply a named object. When unserializing, the class must be defined to unserialize completely, otherwise you’re stuck with an instance of __PHP_Incomplete_class. Hint: when in the same file, class definitions work just like function definitions, i.e. you can put them anywhere, before or after target code.

Notice the Fatal Error above. This example didn’t follow my instructions of having the class “X” defined. To remedy this, simply:

And now, with both PHP snippets merged into a single file, this is what the output looks like:

 

“Falsy” vs “Falsey” – I’m Making the Call

This is probably the most asinine blog post I’ll write, but for my sanity, it must be done.

It is spelled falsy.

Without the “e” between the “s” and the “y” – priod, th nd.

Okay. So what is Falsy anyway?

In loose-typed languages like PHP or JavaScript values can be considered Boolean false if they are '' (empty strings), the number 0 (zero), NULL, and most obviously Boolean false itself, just to name a few (in PHP an empty array is considered false, while in JavaScript it is not). It is important to note that this is only in the context of non-strict comparison, i.e. == and not  ===, and when type casting to Boolean (see PHP’s type juggling reference).

When describing a false-like value it is easier/faster to type “falsy” (or the now incorrect way of “falsey”) since it has less keystrokes, and it is easier to read due to the lack of character bloat; incidentally it also fits nicely as a variable because there is no hyphen (devil’s advocate: camelCase) and it is not a reserved word. It is important that the developer does not simply describe the value as “false” since this implies a variable of type Boolean, however using “falsy” implies a false-like value of any data type which, albeit more ambiguous in its definition, is more descriptive in its context.

Supporting Data

Let’s let the data speak for a moment. After analyzing a Google Trends chart of “falsy” vs “falsey,” the results are initially a bit misleading. “Falsey” is the clear winner at first glance, only to find that there is a name and published articles with the name “Falsey.” Considering that “falsy” is not trailing by many datapoints provides the confidence that it is in use and very applicable. Most importantly in regards to the related searches, “falsy” prevails over “falsey” which does “not [have] enough search volume to show results”: “javascript falsy” and “falsy values.”

Addition: Douglas Crockford even uses “falsy” in his writing, “The Elements of JavaScript Style.” Perhaps I didn’t need to write this post after all, but I sure hope it solidifies the answer indefinitely and provides a resource for any other concerned developers out there.

Spelling Review

I am by no means an English major, or anywhere near one, but I do have a knack for research and comprehension.

With regard to the suffixation of the letter “y” (vowel) when transforming a word, the “e” is dropped per the rules:

In the word “false,” the word ends with the vowel “e,” is preceded by the consonant “s,” and none of the exceptions are applicable.

F – A – L – S – Y