Wednesday, November 9, 2016

Handling messages forwarded as attachment by Outlook with MIME::Parser in Perl

Outlook sends emails that are forwarded as attachment with an .eml extension and the content-type set to application/octet-stream. According to RFC 1341, message/rfc822 should be used. The Perl module, MIME::Parser will automatically parse message/rfc822 attachments, which is useful if you want to do automated processing on an email and its attachments. Outlook's use of application/octet-stream breaks this.

It is possible to fix this. I initially attempted to change the contetn-type and rerun the parser on the file, but that resulted in an empty part. The problem is that according to RFC1341, the Content-Transfer-Encoding field must be 7bit, 8bit or binary for message/rfc822 (Outlook uses base64). Once this is corrected, it works.

A Perl sample: (in this case, the email forwarded as attachment is the second attachment)
#!/usr/bin/perl

use warnings qw(all);
use MIME::Parser;
use strict;

my $fn = '/tmp/input_file.eml';

my $parser = new MIME::Parser;

$parser->output_to_core(1); # Disable the creation of temporary files

my $entity = $parser->parse_open($fn);
$entity->dump_skeleton;   # View initial structure

# Fix the fields
$entity->parts(1)->head->replace('Content-Type','message/rfc822');
$entity->parts(1)->head->replace('Content-Transfer-Encoding','7bit');

# Get encoded message
my $message = $entity->as_string;
#Re-parse
$entity = $parser->parse_data($message);

$entity->dump_skeleton;          # show final structure

\

Here is a general function to handle these. It uses undocumented interfaces, since there does not seem to be a documented method to replace a part with another one.
sub handle_forwarded_messages
{
   my($parser,$entity, undef) = @_;
   return undef unless ($entity && $parser);

   my($part);

   # Recursively process multipart entities, based on number of parts
   if (scalar $entity->parts) # If we have sub-parts
   {
      # Warning, next line uses undocumented interfaces..
      for (my $i = 0; $i <= $#{$entity->{ME_Parts}}; $i++) {
         $part = $entity->{ME_Parts}[$i];
         # Warning, next code line uses undocumented interfaces..
         # Replace part with its expanded version... This seems to be the only way
         $entity->{ME_Parts}[$i] = &handle_forwarded_messages($parser,$part);
      }
   } else { # Once we are at a level that does not have sub-parts...
      # Replace forwarded messages with properly expanded versions...
      if ($entity->effective_type eq 'application/octet-stream' &&
              $entity->head->recommended_filename =~ /\.eml$/) {
          $entity->head->replace('Content-Type','message/rfc822');
          $entity->head->replace('Content-Transfer-Encoding','8bit');
          my $entity_tmp = eval { $parser->parse_data($entity->as_string) };
          $entity = $entity_tmp unless ($@ || $parser->results->errors);
          # And see if they have more levels...
          $entity = &handle_forwarded_messages($parser,$entity);
      }
   }
   # Return the processed result
   return $entity;
}

Wednesday, June 22, 2016

Marking files on UFS as compressed on Solaris

Bernd Schemmer has an interesting post about using fiocompress for file-system level compression of individual files on UFS file systems...

However, it might be useful to mark files as compressed after compressing the file, such as when you forgot the "-m" option when compressing a large file.

The fiocompress utility does this by calling the _FIO_COMPRESSED ioctl on the file.

There seems to be no way to unmark a file that is marked as compressed. (The ioctl sets a cflag on the file called ICOMPRESS, but no operation to clear the cflag seems to exist)

I stripped down fiocompress to a minimal tool to just mark a file as compressed. It is important to ensure that it is a valid file (outfile in the example) generated by "fiocompress -c infile outfile" before running this command. Bad things may happen if this is not the case.

To mark output as compressed, compile the code (compile.sh should do that for you) and run "./markcompressed -m /path/to/outfile" if you are running it from the directory where it was compiled.

Source code can be downloaded from here. (A quick ugly hack, based on fiocompress from OpenSolaris, with some likely bugs)

As always, you should ensure that you have the latest recommended patchset installed.

Older OpenSSH versions - using a different authorized_keys file for a single user

Sometimes, multiple users share a home directory or there are other reasons why a user's authorized_keys file won't pass OpenSSH's permission checks. While those can be disabled, it is generally a bad idea.

In recent OpenSSH versions, using a different authorized_keys file for a single user is easy:

Match User username
AuthorizedKeysFile /etc/ssh/authorized_keys/%u

In older versions however,AuthorizedKeysFile is not supported under a Match block.

In order to get around that, at least on some version, you can use AuthorizedKeysCommand instead, which is supported under a Match block (at least on an up-to-date RHEL5): (It can also be used outside a Match block to enable the option for all users)

Match User username
AuthorizedKeysCommand /etc/ssh/etc_ssh_authorized_keys

You can then create the following script to use as the command and put keys into /etc/ssh/authorized_keys/username #!/bin/sh
/bin/cat "/etc/ssh/authorized_keys/${1}"

To prepare the script and directory: (Users can view each other's public keys in this example, but that for a small number of users is a smaller risk than disabling the permission checks, which might allow other users to edit the user's authorized_keys file)

echo -e '#!/bin/sh\n/bin/cat "/etc/ssh/authorized_keys/${1}"' > /etc/ssh/etc_ssh_authorized_keys
mkdir -m 755 /etc/ssh/authorized_keys
chmod 755 /etc/ssh/etc_ssh_authorized_keys
chmod 644 /etc/ssh/authorized_keys/* # there is probably nothing there yet
chown -R root:root /etc/ssh/etc_ssh_authorized_keys /etc/ssh/authorized_keys

Wednesday, May 4, 2016

Comparing FreeBSD pkgng package options between repositories

When switching FreeBSD repositories, it can be tricky to find which options changed in the package configuration.
This set of bash functions might be useful:
# This will list the options for the package matching the first parameter in the repo given by the second parameter
options ()
{
    sqlite3 /var/db/pkg/repo-$2.sqlite "SELECT p.name,o.option,po.value FROM option o, packages p, pkg_option po WHERE po.package_id = p.id and po.option_id = o.option_id and p.name like '$1' ORDER BY name,option"
}

# This will show the difference between the myrepo repo and the FreeBSD repo for the package fiven as a parameter. This can be changed for any repos or repos from parameters
options_diff ()
{
    diff -u <(options "$1" myrepo) <(options "$1" FreeBSD) | grep '^[+-]'
}

Thursday, January 14, 2016

Configure more secure SSH algorithms on Cisco IOS



Security checkers, like Nessus, often report issues like these on Cisco IOS devices:
  • SSH CBC Mode Ciphers Enabled 
  • SSH Insecure HMAC Algorithms Enabled 
There have been a feature request to add the functionality to IOS, that seems to have been resolved in January 2016.

In the versions where it has been resolved, you should be able to:
> enable
# configure terminal
(config)# ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
(config)# ip ssh server algorithm mac hmac-sha1

You might want to check with "?" if better options have since become available, especially from the MACs (SHA-1 is not ideal, SHA-2/SHA-3 based algorithms might be added in the future) before using my list as-is...

Source of config syntax: Cisco IOS SSH configuration guide

Tuesday, December 1, 2015

Working around Corrupted MAC on input. with Cygwin SSH

When connecting to some servers from Cygwin using ssh, the connection fails and I get this error:
Received disconnect from 1.2.3.4: 2: Corrupted MAC on input.

Looking at debug output, it seems like it is using "umac-64@openssh.com" as MAC algorithm. Forcing it to use something else works around the problem.

When using ssh directly, you can use this: (Other options should work as well, this one worked for me)
ssh -o 'MACs hmac-sha1-96' user@host
or
ssh -m hmac-sha1-96 user@host

When using rsync, you need to put one of the strings above in the --rsh parameter:
rsync --rsh="ssh -m hmac-sha1-96 user@host" :/remote-source /local-dest

A more permanent option is to add the setting to the ssh_config file. This is either /etc/ssh_config (affecting all users) or ~/.ssh/config.

A line using a modified set based on the defaults for my SSH version is:
MACs umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,hmac-ripemd160,hmac-sha1-96,hmac-md5-96

You can find the problematic MAC by running 
ssh -v user@host-that-disconnects-you 2>&1 | grep mac

Adding that as a line to ~/.ssh/config seems to be the easiest solution.

Monday, July 7, 2014

A set of wideband antennas for the FunCube Dongle Pro(+), hackRF, bladeRF and RTL-SDR

When I got my Funcube Dongle Pro, I was looking for a suitable set of antennas.

Since it covers such a wide bandwidth, finding a single antenna that covers the entire bandwidth is hard.

Log-periodic antennas seem to be the most suitable wideband antennas. (They are however directional, which can be good or bad, depending on the application) However, they get impractically large, and as a result unobtainable for frequencies below about 400MHz. For these frequencies, the only practical options seem to be telescopic antennas (which can be tuned for a frequency by differing the length that is expanded) or scanner antennas.

I ended up ordering this set:

I have since acquired a RTL-SDR dongle. The dongle that I have has a TV-style 75-ohm input connector, as opposed to the 50-ohm SMA connector on the Funcube dongle. Ideally, a balun would be used to match the impedance, but as a minimum, some cables can be made up with the relevant connectors at the ends. I found this video with information on building your own, if you need the increased sensitivity that a properly matched antenna provides. Some other dongles, like the one from hakshop, a random one from Amazon and the NooElec one has MCX connectors, which seem to mostly be 50-ohm.

I have also ordered a hackRF, which has a higher maximum frequency, which requires another antenna. I'm planning to acquire this one. (The ANT500 antenna that is available for it seems like a suitable replacement for the telescopic antenna). My planned antenna set for it:

Another transceiver that is available is the bladeRF. It covers 300MHz - 3.8GHz, but it has a higher sampling rate than the hackRF (and a FPGA). An upconverter that allows for lower frequencies are also available. A nice antenna set for it is:
For the low end of the range, if you have the space, also look at the scantenna. It covers 30MHz - 1.3GHz (with several gaps). (It might not be safe for transmitting (bladeRF and hackRF)). (Full specs)

This discone also seems like a great antenna if you can install an outdoor antenna. It can handle transmitting up to 200W on the 6m, 2m, 70cm, ~1.3GHz ham bands. Reception is listed as 25MHz - 1.3GHz


I would also recommend a full set of converters, covering different types of antennas. Some examples: