FAQ's - Rules (Permission Management) ( 15 Articles )
Frequently asked questions about ChoiceMail rules or permission management.
What kind of functions are available in permission rules?
ChoiceMail One supports some special keywords that can be used to match messages whose content has been deliberately distorted by spammers to try and prevent filter systems from recognizing them as spam.
The keywords are
(not)
(me)
(email address in whitelist)
(no email address in whitelist)
(seq)
(word)
(phrase)
(regex)
See http://www.digiportal.com/support/choicemail/onlinehelp/specialkeywords.htm
for more information
Managing forged self-messages
Solution One oft-used tactic by spammers is to fake the FROM address so that it looks like a message you received was actually sent out by you.
By default, ChoiceMail detects and deletes such messages. It does this by adding a code to any outgoing messages that you address to yourself and then deletes incoming messages that don't have that extra information in them.
However, some vendors and some automated systems sometimes send email to you using your own address as the sender even though of course you are not the sender. As an aside, we suggest that you complain to any vendor that does this. The fact of the matter is that they are forging your email address (even if they are doing it with the best of intentions). Ultimately, they will have to change this mechanism as systems such as sender-id and SPF will prevent such messages from being received by your ISP.
In the meantime, there are several ways you can deal with this issue if you really need to receive such messages. The best approach of course is to give vendors one of your virtual email addresses (available only in ChoiceMail One) because such messages will be automatically allowed through without challenge.
If you do not in general receive spam that uses your own address, then you can switch off ChoiceMail's detection from the Options | Preferences menu.
A third way to handle it is to create a permission rule that looks for something else in the header from the vendor or system that is using your email address and use the detection of that header to "validate" the message and let it through.
For more information about permission rules and how to create them, please see this article in the online help.
Permission management
You can add rules to ChoiceMail to automate the approval or rejection process based on the address or content of an incoming email message.
Note that unlike most systems where rules are used to control the disposition of all messages, rules are only needed in ChoiceMail under exceptional circumstances. For example, if you have a subscription to a newsletter or you receive messages through a listserver, and the sender address is different for each incoming message, you can create a rule based on some other part of the message that will be common to all messages from the same source.
You will rarely want to add domains to the "Accepted domains" list unless you are sure that spammers will not be able to spoof those domains (which typically only happens if the domain is not known outside your own network).
The "Rejected domains" list is slightly more useful. Although you normally don't need to worry about individual messages from new senders because they won't get through to you without registration, you may decide that you don't even want to send a registration request to senders from certain domains.
You can tell ChoiceMail One that it should reject all mail whose address matches a particular domain. Although it is not strictly necessary to do this from a spam prevention perspective, rejecting certain domains known for being originators of spam will reduce the number of unknown senders that appear in ChoiceMail One. You can access the rejected domain tab by clicking on Actions | Permission Management and then clicking on the Rejected domains tab.
You can add a complete domain, subdomain, or partial domain. It is also possible to use wildcards to specify a group of similarly named domains. Finally, the special name (none) can be used to refer to an empty domain, i.e. a message from an email address that contains no domain. Note that the parentheses around the word 'none' are required.
Examples:
1) Suppose you receive an email from foo@junk.email.com and you would like to block all emails coming from junk.email.com in the future. Simply enter junk.email.com in the edit field and press the Add button.
2) Blocking mail from junk.email.com will not prevent you from getting email messages from the domain spam.email.com. However, rather than adding this new domain to the list, you could remove junk.email.com and replace it with email.com.
3) Now, a minor problem with the above is that it blocks all messages that end with email.com which means that a message coming from yourfriend@myemail.com would also be rejected. If you don’t want this to happen, then change email.com to .email.com (note the period in front of the domain name).
4) Sometimes a spammer using fake domains will use a related group of domains such as email1.com, email2.com, email3.com and so on. In this case it’s clearly not practical to add each domain separately. Instead, the solution is to specify the domain using a wildcard. There are two kinds of wildcards available. The question mark ? can be used to represent any single character. The asterisk * can be used to represent a variable number of characters (including none). So specifying email?.com will block any messages whose domain ends with the name email followed by any single character. A specification of email*.com will block email.com itself as well as domains such as email123.com, emailxyzzy.com and so forth.
Note that ChoiceMail examines the individual rules before checking either the Accepted or Rejected domains lists. This lets you easily define exceptions. For example, you may choose to reject all messages from a particular domain except for those coming from a few particular individuals who are already known to you. Alternatively, you may choose to accept all messages coming from a particular domain except those from a few particular senders whose messages you don't want.
The actual order in which messages are processed through the permission system is as follows:
1. Whitelist/blacklist
2. Rules (in the order in which they appear in your message rules list)
3. Forged self-message check
4. Rejected domains
5. Accepted domains
The color of a rule is intended as a quick reminder of its purpose. Rules in red indicate they will delete or reject messages. Rules in green indicate that they will accept messages or senders. Rules that are in BOLD mean they were created by you as opposed to those that were included in the base product. You can select a rule and press the Explain button to get more information about the selected rule. The built-in rules are normally hidden unless you check the "View built-in rules" checkbox. This means that when you open the permission rules view for the first time, it will not display any rules in it - you're free to ignore the built-in rules and just add in a few of your own if you need them. Many people don't need to add ANY extra rules except perhaps to handle some newsletters or online vendors.
Click New to create a new rule or click Modify to edit the selected rule.
Apply: Click apply to apply all the rules to your current list of unknown senders. This process can take quite a long time, depending on the number of messages and complexity of particular rules.
Test permissions: this button opens a dialog where you can fill in different parts of a form that represents different parts of an email message. As you fill in the fields of the simulated message, the dialog will tell you which permission rule would match the message.
See also:
Creating or modifying an email rule
Accepted and rejected domains
IP blocking
Testing your rules
Messages are being deleted by the URL with digit portion or % permission rule
Symptom: you are finding that some valid messages are being deleted by ChoiceMail and the reason specified in the junkbox is "URL with digit portion or %"
This permission rule is intended to detect attempts to disguise a URL by using digits or % signs rather than a legitimate server name.
However, the latest version of Panda Internet Security inserts a "this is not a spam" tag at the end of each incoming email. Their tag includes a reference to http://127.0.0.1 and this is what is being detected by ChoiceMail.
Solution:
Disable spam checking in Panda (you've already got ChoiceMail!)
I am having trouble getting permission to send someone an email
If you are a trying to send a message to someone whose emails are being protected by ChoiceMail, you must fill in a form at a webpage with your reasons for wanting to contact the user.
You will have received an email message from that user's ChoiceMail system containing a URL on which you should click to get access to the form.
Unfortunately, some email applications break the URL up into multiple lines and only leave the first line "live". Therefore when you click on the URL, only the beginning of the URL is being used by your browser.
We have provided an unsupported tool that can repair the damage in SOME cases - we can't guarantee it will always work - it really depends on how badly the email program broke the link.
Click here to download a utility called RepairEncryptedURL.exe from our website and run it. A small window will be displayed into which you can then drag your broken URL. If the program can repair the URL, it will copy it to the clipboard so that you can paste it back into the address bar of your browser (Mozilla, Firefox, Internet Explorer, Netscape, etc) and try again.
If you find that the registration process does not accept the code you typed in, it is possible that your browser has cached the image and so has an "old" code displayed. You can test this by explicitly reloading the registration page using your RELOAD button. The value in the code box should change each time you do this. If it does not change, then your browser is not correctly reloading the registration page. Some older versions of Netscape are particularly prone to this problem.
Some senders don't bother to fill in the permission form. Will I still get their mail?
That's up to you. Mail is held temporarily by ChoiceMail until someone registers and you can always choose to review held-up mail manually (ChoiceMail makes it easy to do that quickly).
On the other hand, many people take the position that if someone doesn't want to bother to fill in the form (a simple brief one-time effort for anyone not already on your whitelist), then you probably don't need to see their mail anyway. After all, if the sender doesn't consider their message to be important enough that they need to make sure you read it, why should you consider it worth reading!
Using regular expressions in permission rules
Warning - DigiPortal Software does not provide technical support on this topic and all requests will be quietly ignored. The information in this document is provided as-is and intended only as a starting point for advanced users familiar with the concepts described below.
ChoiceMail One supports the use of regular expressions to specify match conditions in permission rules. An explanation of regular expressions is beyond the scope of this document but you can search for the phrase using your normal search engine to find more information on this topic. A particularly interesting website is http://www.regexlib.com/ which contains many examples of regular expressions. Microsoft also has an excellent regular expression tutorial on their website.
To use a regular expression instead of simple text, start the line with the sequence
(regex)
Everything immediately following that sequence will be treated as the regular expression.
Example:
(regex)(?i)v[i|1]agra
* (?i) A switch that means that the match is not case-sensitive
* [i|1] matches either the letter i or the number 1
So the expression above will match viagra, v1agra, ViAgRa, and so forth.
Example:
(regex)(?i)</html>[\s]*(<.*>)*?[a-zA-Z0-9]+
Placed in the BODY section of a permission rule, this expression does the following:
1. (?i) A switch that means that the match is not case-sensitive
2. </html> Looks for the normal end tag of an HTML document ,followed immediately by
3. [\s]* Any number of blank spaces, followed immediately by
4. (<.*>)*? An arbitrary number of characters (including none), followed by
5. [a-zA-Z0-9]+ At least one alphanumeric character. The uppercase range A-Z is not actually necessary due to the (?i) switch but is good practice since the switch is not always used.
Comment: a properly formed HTML document should never have characters after the final </html> tag. Spammers often insert random characters (different in every message) so that server-based filtering system will not recognize the messages as spam (filter systems often presume that if the identical message is received many times, then it's highly likely to be spam).
How do I switch off all the permission rules?
ChoiceMail comes with a collection of built-in rules that are intended to delete messages that it can be certain are spam. The goal is to reduce the number of unknown senders that show up in ChoiceMail and reduce the number of challenges that need to be sent out.
For example, legitimate unknown senders should never send you messages containing URLs with disguised server names. So we include a permission rule that detects that URLs have been disguised and use that as a criterion for deletion.
However, like everything about ChoiceMail, this process can be tailored to suit individual customer needs.
Built-in rules are hidden by default but all you need to do to see the built-in rules is to check the "View built-in rules" box in the permission management dialog. You can then reorder them, disable them or delete them as desired.
If you are in the permission management dialog, you can just press F1 for online help to see more information on this topic.
How do I disable ALL permission rules without opening ChoiceMail?
Please do the following.
1) Make sure ChoiceMial is NOT running. (for V4 and up make sure you open up the ChoiceMail Control Panel and STOP the service)
2) Go into the folder called \Rules located under the main ChoiceMail folder.
3) Find a file that has your username or email address and end with .xml
4) Open this file in notepad. (Right click on it and select open with and then selct notepad)
5) In Notepad select Edit->Replace and in the "find what" box type in
enabled="1"
For the "replace with box" type in
enabled="0"
(note that we want to change all occurrences of enable="1" to enabled="0")
6) Save the file and close notepad making sure you have made the correct changes.
7) Start ChoiceMail again.
All the rules should still be visible, but disabled.
Test permission rules
You can test your rules to make sure they are responding properly to incoming messages. Simply type text into any of the fields of the Test Rules dialog and the results will be displayed when you stop typing for a few seconds, indicating which rule, if any, responded to your entry.
Transform Rules
Warning - do not attempt to change the contents of the transformrules file unless you really understand what you are doing. DigiPortal Software does not provide technical support on this topic and all requests will be quietly ignored. The information in this document is provided as-is and intended only as a starting point for advanced users familiar with the concepts described below.
As of ChoiceMail One version 2.5, the transform rules file is edited through a tree representation rather than through the raw text file.
Email messages can be preprocessed before any tests are applied to them. This is particularly useful to get rid of bogus HTML tags that are often inserted into email messages in an attempt to prevent undesirable keywords from being detected.
For example, suppose you have noticed that you're getting a lot of messages containing the word 'viagra' and you'd like to create a rule to delete any incoming message that has the word 'viagra' in it. However, attempts to create a standard permission rule with 'viagra' as the keyword in the body section are failing.
The reason it's not working is because in the underlying email message, the word 'viagra' does not appear by itself. Instead, it will be broken up by using bogus HTML tags. Here is a real example from an email message recently received:
G<!--w88u0k1fg2nm4o-->et Vi<!--06qso523nnr-->a<!--or55j23i3u-->gra o<!--942m1c3t4eto-->nline
The 'HTML' tags are bracketed with '<' and '>' but they are in fact invalid tags. Most browsers will simply ignore them, thereby displaying
Get Viagra online
Standard keyword searching will however fail to find 'viagra' anywhere.
The transformrules.ini file uses a search/replace mechanism that can be used to temporarily change a message (the subject and/or the body) so that it is easier to find certain keywords. However, unlike the basic search/replace mechanism that is found in most word processors, the transform rules subsystem uses a mechanism called regular expressions to allow more general matching. An explanation of regular expressions is beyond the scope of this document but you can search for the phrase using your normal search engine to find more information on this topic.
Format of the transform rules file
The file is divided into named sections. Each section is enclosed in square brackets. A section is associatred with a particular permission rule by specifying the section name as the value for the "Message preprocessor name' in a permission rule editor. Any line that starts with a semicolon is considered to be a comment and will be ignored by the system.
Within a section, you can have as many transform rule groups as you like and each rule group will be applied to the result of the previous transformation in the section.
A rule group consists of three parts:
regexN=....
targetN=...
replaceN=...
where N starts at 1 and is incremented by 1 for each successive rule group. The values for the regexN and replaceN keywords will be single regular expressions. An expression is required for the regexN keyword but is optional for the replaceN keyword.
The value for the targetN is either body or subject meaning that the group should be applied to either the body or the subject of the message respectively.
As each rule group is processed, all text in the message matching the value of the regexN expression will be replaced by the content of the associated replaceN expression. All other text is left intact.
Example:
; This section just strips all HTML tags out
; of a message and then deletes all spaces.
; It makes it easier to look for bad words
; that the spammers have attempted to hide
[Strip HTML]
;First, eliminate all HTML tags from the body
regex1=<.*?>
target1=body
replace1=
;Now get rid of all spaces
regex2=(\ )\s*
target2=body
replace2=
You can edit your transform rules by clicking on the Edit button in the email rule dialog to open up your existing rules. If you do not have any transform rules, you can start by pasting the example above into the transform rules window.
The email rule dialog example also shows how this transform rule can be used with a regular expression to eliminate a substantial number of unwanted emails.
Use the following regular expression in the BODY section of a rule that uses the transform rule above to preprocess the received email message.
(regex)(?i)v[i|1|í]agra|p[e|3]n[i|1|í]s|p[e|3]n[i|1|í]le
This expression looks for the words viagra, penis, or penile in the message after it has been preprocessed, taking into account that spammers often try to disguise these words by changing one of the characters in them to something else.
100% CPU utilization
If you are experiencing 100% CPU utilization, it may be due to one of the permission rules that is included with ChoiceMail One version 2.0. The rule is called "Stuff after end of HTML page" and under some rare circumstances, it can cause a large amount of processing that will tie up your system.
You can switch off this rule. Perform the following steps:
1) Click on Actions | Permission Rules
2) Scroll down the list of message rules until you find the rule called
Stuff after end of HTML page
3) Uncheck the box on the left
4) Click OK
A message I wanted was deleted by the Domain Address Check permission rule
The \"Domain Address Check\" looks in the body of a message for URLs that are from servers around the world known to be spam sources. These servers are collected into regions in our system.
So one of the URLs in the body of your message matched one of those domains. If in fact you get legitimate mail containing URLs from one of those regions, you may wish to remove that particular region from the list.
To do this, you have to do one of two things
A) Disable the Domain Address Check rule compeltely
Doing this is easy but means that more spam will show up in your
unknown senders list (which may or may not matter to you) and more
challenges will be sent out.
To do this, follow these instructions
1) Open the Permission Management dialog (Actions | Permission
management)
2) Click on the Rules tab
3) Click on the checkbox called \"View built-in rules\"
4) Scroll down the list of rules to the bottom where you should find a rule called \"Domain Address Check\"
5) Click the checkbox to the left of that rule to disable the rule
6) Click OK to close the permission rule dialog
B) A more sophisticated approach is to eliminate the specific IP address that is causing the problem.
1) View the message in the Safe Message Preview window
2) Switch to the Domain List view where you will get a list of all domains found
3) Right-click on each domain and select \"More info on...\". This will open a web browser window and provide you with information, including the IP address of each URL that was found
4) Open the permission management dialog
5) Click on the \"Advanced\" tab
6) Click on the \"Configure...\" button - you will get a dialog telling you how many address ranges are blocked
7) Click on View | Advanced in the menu of that dialog
You will see a list of IP addresses from various places.
8) Find the IP address range that incldules the address you found from (3) above
9) Click on that IP Address rnage to select it
10) Click the Delete button to remove it
11) Click OK to close the dialog (and click OK on the other dialogs that were opened along the way)
How can I automatically delete any mail that is not explicitly addressed to me?
Solution You can do this by creating a permission rule that uses a special keyword to check whether the address in the TO field belongs to you (i.e. it exists as a POP3 account or as an alias). This rule will also delete messages that have NOTHING in the TO field. Note that if your addess is not found in the TO field, it is treated as a BCCd message.
1. Click on Actions | Permission Management...
2. Click on the Rules Tab
3. Click New... to begin creating a new permission rule
4. Click on the "To" tab
5. Enter the phrase (not)(me) exactly as written, including parentheses and with no spaces
6. Change the text in the Rule name field to indicate what you have done. For example, call it "Delete messages not addressed to me"
7. Click on the Actions tab
8. Make sure that the option "Delete sender to junkbox" is checked
9. Click OK to close the New Email Rule dialog
10. Click OK to close the Permission management dialog
You now have a new rule that will delete any message that does not explicitly have your email address in the TO field.
Firewalls and server permissions
If you are using a software firewall such as ZoneAlarm, you may be prompted several times for client and server permissions when you run ChoiceMail 1.5 the first time.
The three components are
1. ChoiceMail
Requires permission to act as a client and as a server. Unless you are planning to access ChoiceMail from other machines, you can restrict server permissions to your local trusted zone.
2. ChoiceMail WebGate (also known as IzyMail)
Requires permission to act as a client and as a server. The server permissions can always be restricted to just local machine access.
3. True Update (also known as cmupdater)
Requires client permission only.
Alternatively, ChoiceMail One may just not be able to connect to the Internet and you get a socket error such
10061 - Connection Refused
which essentially means that your firewall is silently blocking ChoiceMail One.
Note that this is NOT a problem with ChoiceMail One itself - you may need to contact your firewall vendor if you continue to see this error.
For the technically minded, by default ChoiceMail One listens for your email client using port 110, the standard POP3 port and on port 25, the standard SMTP port. It is only necessary that these ports be open on your trusted machine. If you have a hardware firewall, these ports are typically blocked from outside access.
If you only have a software firewall such as ZoneAlarm, you can configure it to only allow those ports to be opened from within your trusted network (which normally just means your own machine unless you're running a home LAN) and access will continue to be blocked from the outside world.
If you're not running a hardware or software firewall, stop what you're doing right now and go get one, even if you are not using ChoiceMail One! Running a PC these days without at least a software firewall leaves your PC at risk to serious attacks from malicious hackers.
(DigiPortal Software cannot help you to choose a firewall or provide help configuring your firewall - for the latter, please contact your firewall vendor).
Note that you can change the ports used for POP3 and SMTP between ChoiceMail and your email client if you wish - but make sure they match or else your email client will not be able to communicate with ChoiceMail.
Customers have also reported problems with ChoiceMail refusing to work or generating access violations after they convert to the retail version. This is usually due to the software firewall not giving permission to ChoiceMail. Please check your firewall permissions very carefully.