Newsletter VS the spam filter.

Ugh.

Over the last while I have been putting together a newsletter service for my photography website.  As with most development, a lot of work went into this to get something simple at the surface level to work just right.  And, again as with most development, unforeseen headaches reared their ugly heads as soon as it was pushed live.

This newsletter service was no exception.  For some email addresses, everything worked perfectly as it should.  For others, confirmation emails were flagged as spam and hidden away in a background folder.  And others wouldn’t receive a darn thing (mysteriously sent e-mails couldn’t be found in either the inbox or the spam folder).

Where to start in debugging this.  Well, if you happen to be going down the same path I was, take my advice and utilize this useful website:
www.mail-tester.com

Simply send it a sample e-mail and it will analyze it in the same way that other email providers will when looking for spam.  Does your subject resemble a lot of known spam subjects?  It’ll warn you about that.  Forgot to set a Sender Policy Framework (SPF) key in your DNS?  Yup, this will remind you too.  DKIM missing?  Bounce e-mail spoofed incorrectly?  Yup, it’ll alert you to those too.

Basically it helps you fix the problems that others might see with your otherwise awesome email.   Problems you might now be aware of otherwise.

Don’t get too frustrated, there’s always a solution! 😛

Springpad to Evernote Migration

Evernote is one of those programs that I had heard a lot about peripherally over the years but never really dived in to explore it myself.  The pile of documents in my Dropbox account however was beginning to get a bit unwieldy, and the traditional file system model just wouldn’t allow me to make the additional links of logic that keywording organization could, so I thought I would finally take a look.

So far I have enjoyed the new-to-me ecosystem, and have made the move to consolidate a few of my other accounts into my Evernote one.  Springpad in particular seems to be one that people are having problems migrating over.

I have written a Python script that saved me and a friend some time, and maybe it can save some of you some time as well.  It’s not perfect, and it takes some organization/editing  once everything is imported to Evernote (due to different APIs), but it is a good starting point.

#!/bin/python

import re

def writeNote(noteSubjectParam):
  origParam = noteSubjectParam

  for ch in ':;/\\`<>|' :
    noteSubjectParam = noteSubjectParam.replace("&", "&")
    noteSubjectParam = noteSubjectParam.replace("&#8211;", "-")
    noteSubjectParam = noteSubjectParam.replace(ch, '-')

  print "Processing: ", origParam, "\n         -> ", noteSubjectParam

  nf = open(noteSubjectParam + '.htm', 'w')
  nf.write(note)
  nf.write(footer)
  nf.close()

f = open('index.html','r')
header = ""
headerComplete = False
footer = """</div>
</body>
</html>
"""
noteStart = False
noteSubject = ""
note = ""
h2re = re.compile('
<h2 class="fn">(.*)</h2>
', re.IGNORECASE)
invalidFSChars = re.compile
i = 0

for line in f:
  if '
<div class="instance' in line:
    if not headerComplete:
      headerComplete = True
      #Finish header
    if noteStart:
      #Finish note
      writeNote(noteSubject)
    i += 1
    noteStart = True
    note = line
  elif '
<h2 ' in line:
    m = h2re.match(line)
    noteSubject = m.group(1)
  else:
    if not headerComplete:
      header += line
    elif noteStart:
      note += line

if noteStart:
  #Finish last note
  writeNote(noteSubject)

General usage instructions:

  • Save this code to a Python file (eg: pythonSplitScript.py) in its own directory.
  • Download a backup of your Springpad data.  Log in to your account at www.sprintpadit.com and go Settings -> Services -> Backup to create/download a zip file of your account information.
  • Extract the zip file and you should see an index.html file.  This file has all of your information in it!  Move this file into the same directory as your script so that it can be split into seperate html files that Evernote can understand and import.
  • Open your terminal and run the script from the command line.  On a Mac, the command looks like:
    python pythonSplitScript.py
    
  • Now there will be a bunch of *.htm files in your directory.  Open your Evernote desktop app, select all these generated files (omit the original index.html file) and drag them into an Evernote notebook to import them.

This should work fine with the Mac and PC Evernote apps.  As far as I know, the mobile and browser apps do not have this import functionality.