Wednesday, November 9, 2016

Handling messages forwarded as attachment by Outlook with MIME::Parser in Perl

Outlook sends emails that are forwarded as attachment with an .eml extension and the content-type set to application/octet-stream. According to RFC 1341, message/rfc822 should be used. The Perl module, MIME::Parser will automatically parse message/rfc822 attachments, which is useful if you want to do automated processing on an email and its attachments. Outlook's use of application/octet-stream breaks this.

It is possible to fix this. I initially attempted to change the contetn-type and rerun the parser on the file, but that resulted in an empty part. The problem is that according to RFC1341, the Content-Transfer-Encoding field must be 7bit, 8bit or binary for message/rfc822 (Outlook uses base64). Once this is corrected, it works.

A Perl sample: (in this case, the email forwarded as attachment is the second attachment)
#!/usr/bin/perl

use warnings qw(all);
use MIME::Parser;
use strict;

my $fn = '/tmp/input_file.eml';

my $parser = new MIME::Parser;

$parser->output_to_core(1); # Disable the creation of temporary files

my $entity = $parser->parse_open($fn);
$entity->dump_skeleton;   # View initial structure

# Fix the fields
$entity->parts(1)->head->replace('Content-Type','message/rfc822');
$entity->parts(1)->head->replace('Content-Transfer-Encoding','7bit');

# Get encoded message
my $message = $entity->as_string;
#Re-parse
$entity = $parser->parse_data($message);

$entity->dump_skeleton;          # show final structure

\

Here is a general function to handle these. It uses undocumented interfaces, since there does not seem to be a documented method to replace a part with another one.
sub handle_forwarded_messages
{
   my($parser,$entity, undef) = @_;
   return undef unless ($entity && $parser);

   my($part);

   # Recursively process multipart entities, based on number of parts
   if (scalar $entity->parts) # If we have sub-parts
   {
      # Warning, next line uses undocumented interfaces..
      for (my $i = 0; $i <= $#{$entity->{ME_Parts}}; $i++) {
         $part = $entity->{ME_Parts}[$i];
         # Warning, next code line uses undocumented interfaces..
         # Replace part with its expanded version... This seems to be the only way
         $entity->{ME_Parts}[$i] = &handle_forwarded_messages($parser,$part);
      }
   } else { # Once we are at a level that does not have sub-parts...
      # Replace forwarded messages with properly expanded versions...
      if ($entity->effective_type eq 'application/octet-stream' &&
              $entity->head->recommended_filename =~ /\.eml$/) {
          $entity->head->replace('Content-Type','message/rfc822');
          $entity->head->replace('Content-Transfer-Encoding','8bit');
          my $entity_tmp = eval { $parser->parse_data($entity->as_string) };
          $entity = $entity_tmp unless ($@ || $parser->results->errors);
          # And see if they have more levels...
          $entity = &handle_forwarded_messages($parser,$entity);
      }
   }
   # Return the processed result
   return $entity;
}

Wednesday, June 22, 2016

Marking files on UFS as compressed on Solaris

Bernd Schemmer has an interesting post about using fiocompress for file-system level compression of individual files on UFS file systems...

However, it might be useful to mark files as compressed after compressing the file, such as when you forgot the "-m" option when compressing a large file.

The fiocompress utility does this by calling the _FIO_COMPRESSED ioctl on the file.

There seems to be no way to unmark a file that is marked as compressed. (The ioctl sets a cflag on the file called ICOMPRESS, but no operation to clear the cflag seems to exist)

I stripped down fiocompress to a minimal tool to just mark a file as compressed. It is important to ensure that it is a valid file (outfile in the example) generated by "fiocompress -c infile outfile" before running this command. Bad things may happen if this is not the case.

To mark output as compressed, compile the code (compile.sh should do that for you) and run "./markcompressed -m /path/to/outfile" if you are running it from the directory where it was compiled.

Source code can be downloaded from here. (A quick ugly hack, based on fiocompress from OpenSolaris, with some likely bugs)

As always, you should ensure that you have the latest recommended patchset installed.

Older OpenSSH versions - using a different authorized_keys file for a single user

Sometimes, multiple users share a home directory or there are other reasons why a user's authorized_keys file won't pass OpenSSH's permission checks. While those can be disabled, it is generally a bad idea.

In recent OpenSSH versions, using a different authorized_keys file for a single user is easy:

Match User username
AuthorizedKeysFile /etc/ssh/authorized_keys/%u

In older versions however,AuthorizedKeysFile is not supported under a Match block.

In order to get around that, at least on some version, you can use AuthorizedKeysCommand instead, which is supported under a Match block (at least on an up-to-date RHEL5): (It can also be used outside a Match block to enable the option for all users)

Match User username
AuthorizedKeysCommand /etc/ssh/etc_ssh_authorized_keys

You can then create the following script to use as the command and put keys into /etc/ssh/authorized_keys/username #!/bin/sh
/bin/cat "/etc/ssh/authorized_keys/${1}"

To prepare the script and directory: (Users can view each other's public keys in this example, but that for a small number of users is a smaller risk than disabling the permission checks, which might allow other users to edit the user's authorized_keys file)

echo -e '#!/bin/sh\n/bin/cat "/etc/ssh/authorized_keys/${1}"' > /etc/ssh/etc_ssh_authorized_keys
mkdir -m 755 /etc/ssh/authorized_keys
chmod 755 /etc/ssh/etc_ssh_authorized_keys
chmod 644 /etc/ssh/authorized_keys/* # there is probably nothing there yet
chown -R root:root /etc/ssh/etc_ssh_authorized_keys /etc/ssh/authorized_keys

Wednesday, May 4, 2016

Comparing FreeBSD pkgng package options between repositories

When switching FreeBSD repositories, it can be tricky to find which options changed in the package configuration.
This set of bash functions might be useful:
# This will list the options for the package matching the first parameter in the repo given by the second parameter
options ()
{
    sqlite3 /var/db/pkg/repo-$2.sqlite "SELECT p.name,o.option,po.value FROM option o, packages p, pkg_option po WHERE po.package_id = p.id and po.option_id = o.option_id and p.name like '$1' ORDER BY name,option"
}

# This will show the difference between the myrepo repo and the FreeBSD repo for the package fiven as a parameter. This can be changed for any repos or repos from parameters
options_diff ()
{
    diff -u <(options "$1" myrepo) <(options "$1" FreeBSD) | grep '^[+-]'
}

Thursday, January 14, 2016

Configure more secure SSH algorithms on Cisco IOS



Security checkers, like Nessus, often report issues like these on Cisco IOS devices:
  • SSH CBC Mode Ciphers Enabled 
  • SSH Insecure HMAC Algorithms Enabled 
There have been a feature request to add the functionality to IOS, that seems to have been resolved in January 2016.

In the versions where it has been resolved, you should be able to:
> enable
# configure terminal
(config)# ip ssh server algorithm encryption aes128-ctr aes192-ctr aes256-ctr
(config)# ip ssh server algorithm mac hmac-sha1

You might want to check with "?" if better options have since become available, especially from the MACs (SHA-1 is not ideal, SHA-2/SHA-3 based algorithms might be added in the future) before using my list as-is...

Source of config syntax: Cisco IOS SSH configuration guide