Here is a new workflow:
You type "oa:install" to install a helper script into your /usr/local/bin directory. Make sure that directory exists, is writable by you, and in your path.
Then you can use these commands:
oa:terminal open the topmost terminal window in Alfred
oa:iterm open the topmost iterm window in Alfred
Let me know if this works for you.