Opened 18 months ago

Last modified 2 months ago

#2087 open enhancement

Enhancements to Susimail

Reported by: Reportage Owned by:
Priority: minor Milestone: undecided
Component: apps/susimail Version: 0.9.32
Keywords: susimail UX UI Cc:
Parent Tickets:

Description

Useful enhancements for Susimail:

  • addressbook with ability to add addresses from received mails
  • folder support to allow for inbound filtering and moving of of mail to custom folders, and an additional folder for detected spam

Subtickets (add)

#2169: Susimail: New mail should remain flagged as new until openeddefectassignedzzz
#2173: Susimail should enable 'Cancel' button on compose mail page to abort sending mailenhancementacceptedzzz
#2176: SusiMail > WebMail.java > onbeforeunload (Doesn't work as intended) / Compose.js (not needed)defectclosed
#2459: Susimail, remarks on the Web GUI.enhancementassignedsadie

Attachments (2)

reportage-2087-01.patch (43.3 KB) - added by slumlord 13 months ago.
Patch by Reportage - 1
reportage-2087-02.patch (93.5 KB) - added by slumlord 13 months ago.
Patch by Reportage - 2

Download all attachments as: .zip

Change History (36)

comment:1 Changed 18 months ago by zzz

  • Status changed from new to open
  • Type changed from defect to enhancement

Low priority. We recommend that people use a full mail client such as Thunderbird if they want features like this. We don't have a lot of time to turn susimail into a full-featured client.

comment:2 Changed 18 months ago by Reportage

I'll use this report to aggregate various Susimail enhancement suggestions. So, in addition to folder support, a UI for configuration that offers more than mails per page (#2085), here are a couple more ideas. Some may be low down the priority list, others may offer a reasonable enhancement for very little work:

  • Allow the user to see the full e-mail headers, either with an option that enforces a global pref, or via a 'show/hide headers' combo for individual e-mails. The headers are visible in the raw e-mails, so parsing them shouldn't be difficult.
  • Option to set an automatic response to incoming e-mails to serve as a mail receipt, notification that the recipient is on vacation etc.
  • In situ log viewer (mostly communication with server) to help with debugging issues etc
  • Support for automated (footer) signature. User configures optional signature which is then added to the bottom of outgoing mail.
  • Encryption of received mail directory (#2081)
  • Option to prevent mail from being downloaded (#2085)
  • Add option to automatically encrypt/decrypt emails with gpg and integrate gpg keys into private addressbook
  • Basic html mail support with default block for all remote resources. Option to load .i2p-based resources automatically, and clearnet resources on a per-resource click-to-load basis (or block and display inline placeholder).
  • Option to automatically forward e-mails to remote address based on sender or subject.
  • Add notification "No new mail on server" when checking for mail does not find anything new.
  • Add intermediate page that indicates that connection to the server is in progress after form is processed, to enhance UX and provide better feedback. When connection to server is complete, return to mail list page.
  • As an alternative to gpg, enable automatic encryption in transit and decryption in the recipient's susimail when correct login is supplied (perhaps with an optional supplementary passphrase). Is the java implementation of OTR worth looking at? https://github.com/jitsi/otr4j In essence, internal mails that don't leave the i2p network should be end-to-end encrypted, and e-mails that are sent to clearnet addresses could have optional encryption if the recipient supports it.
  • Enable automatic scanning of e-mails with option to specify command line parameter to AV scanner
  • Assess suitability of 'trees' for server-side encryption https://0xacab.org/riseuplabs/trees
  • Review https://protonmail.com for potential enhancement ideas
  • Consider offering option to encode/decode e-mail body in image via https://github.com/varunon9/Image-Stegano with optional password protection
  • Option to automatically logout user after configured period of inactivity
Last edited 16 months ago by Reportage (previous) (diff)

comment:3 Changed 18 months ago by Reportage

  • Summary changed from Add addressbook and folders to Susimail to Enhancements to Susimail

comment:4 Changed 18 months ago by Reportage

  • Keywords UX UI added; folders addressbook removed

comment:5 Changed 18 months ago by zzz

Thanks. FYI for others, see #2085 for discussions on the overall state of SusiMail? and prospects for improvement.

comment:6 Changed 18 months ago by zzz

Re: folder issue (from OP)

While I was sceptical in comment 1, we do need a drafts folder. Without it, we can't do async transfer to SMTP and try later on failure. To not have a retry feature is pretty bad, especially for large emails that are more likely to fail. And having the POST hang on send instead of queueing it off is bad also.

If implemented correctly, a drafts folder would make the addition of other folders as described in the OP easier.

Also FYI see #1373 for recent progress on fixing the buggy UI handling.

comment:7 Changed 17 months ago by zzz

Adding to list: Email threading. Take impl from Syndie.
Adding to list: Search.

Overall, I would like to hear everyone's opinions of which new features listed above would be most important (independent of estimated effort)

Last edited 17 months ago by zzz (previous) (diff)

comment:8 Changed 17 months ago by zzz

no new msg / x new msgs shown after check mail button, in 79afa604bde5f539e763a9a68a64ebb13ba68ac1 to be 0.9.32-17

comment:9 Changed 16 months ago by zzz

Thread server connection in background, provide notification at bottom that it's in progress, mostly fixes 11th bullet in comment 2 above. In 0dd9398167d5019b86875c231e8ffd18db791883 0.9.33-5. This is probably the last of the major changes for 0.9.34.

Folders (at least for outbox/drafts - see comment 6 above) and background sending tentatively planned for 0.9.35. Most of the rest is lower priority and may depend on identifying a new maintainer. See http://i2p-projekt.i2p/en/blog/post/2018/02/11/high-level-roadmap

comment:10 Changed 16 months ago by Reportage

The background server connection, specifically to check for mail, is noisy, repeatedly generating a notification on every page. Also, requiring that the user manually refresh to check for mail seems like a regression.

The proposal to add an intermediate page with a "checking mail in progress" notification was suggested so that there's some visual feedback on the status of the connection to the server above and beyond a browser tab spinner. This could happily happen in the foreground, with a status report on the intermediate page indicating any communication with the server. eg:

  • Connecting to server pop3.postman.i2p...
  • Connected to server pop3.postman.i2p; checking for new mail...
  • Mailbox contains xxx messages.
  • No new mail on server.

To avoid the manual refresh, why not use an iframe at the top of the page for notifications? That way you can refresh the notifications once the connection is complete without impeding interaction and avoid notification popups that plague every page.

The proposal to add support for mail folders could alternatively be used to log new mails, with an iframed folders sidepanel that automatically updates on receipt of new mail, allowing background server communication without the need for a manual page refresh or noisy notifications.

Otoh, sending mail in the background seems like a definite win, as long as on successful send the user is notified without having to manually refresh the page. A folders sidebar could be used here to indicate unsent mail, again automatically updating once the mail has successfully been sent.

The proposal to add a search function is also a good one, especially useful for large mailboxes. Searching by recipient, subject, body text and attachment, with results parsed as per the mail listing page with the search terms highlighted would be welcome.

Last edited 16 months ago by Reportage (previous) (diff)

comment:11 Changed 16 months ago by zzz

Blocking during the check, especially at first login, was really bad, and we also tended to keep the timeout relatively low so the user didn't give up. Also, we've repeatedly gotten complaints that people couldn't log in without multiple retries, probably because of the short timeout. Doing it in the background allows us to increase the timeout and the success rate, while allowing the user to access local mail. It's like "offline mode" until connected. With this fix, maybe we can remove the "read mail offline" button.

So on balance, this is an improvement, not a regression. An intermediate page would not be better. I do recognize that this change does not meet the exact request in the 11th bullet.

However, I agree that it would be even better to have a popup or section of the page that shows status. You suggest an iframe, but javascript would be the correct way. I considered it but decided it would be too much to do for .34. It could, perhaps, be in .35.

Thanks for your other comments.

comment:12 Changed 15 months ago by zzz

Add a subticket #2169.

comment:13 Changed 15 months ago by Reportage

Another feature request: batch file add for file attachments. If pointed at a directory, Susimail should add all files, optionally compressing them into a single zip container. Or, if multiple files in a directory are selected, do the same.

Last edited 15 months ago by Reportage (previous) (diff)

comment:14 Changed 15 months ago by zzz

Add a subticket #2173.

comment:15 Changed 15 months ago by zzz

Idea: "linkify" the URLs in the displayed text/plain message bodies. This would make reading that version (while hiding the html alternatives) much less painful.

The change would be a big update to - or fancy replacement for - EscapeHTMLWriter. Not easy to parse safely and on-the-fly but doable.

comment:16 Changed 15 months ago by zzz

More fixes for background email checking and result messages (#2174, comments 8,9,10,11 above) in 0.9.33-10

comment:17 Changed 15 months ago by zzz

Add a subticket #2176.

comment:18 follow-up: Changed 14 months ago by zzz

Folders and background sending (comment 6 above) mostly complete for 0.9.35 but not checked in.
If there's anybody who wants to do alpha testing now, let me know and I'll check it into a separate branch. Otherwise I plan to check it into trunk early in the 0.9.35 cycle, mid-late April.

Edit: anybody requesting to test must affirm they have the ability to pull from Monotone and build from source. The branch, if created, would not be available on Github, and would not be built by any build bot.

Last edited 14 months ago by zzz (previous) (diff)

comment:19 in reply to: ↑ 18 Changed 14 months ago by slumlord

Replying to zzz:

Folders and background sending (comment 6 above) mostly complete for 0.9.35 but not checked in.
If there's anybody who wants to do alpha testing now, let me know and I'll check it into a separate branch. Otherwise I plan to check it into trunk early in the 0.9.35 cycle, mid-late April.

Edit: anybody requesting to test must affirm they have the ability to pull from Monotone and build from source. The branch, if created, would not be available on Github, and would not be built by any build bot.

BobTheBuilder?.i2p is now tracking mtn and can be set up to build and distribute any branch in the I2P monotone database.

comment:20 Changed 14 months ago by zzz

Idea: New mail indication in console summary bar (not easy)

Idea: In-router API for sending mail to self or others, e.g. for automated reports (not easy)

re: folders (comment 18), no request received for a branch. Will be checked in to trunk, est. wk of April 16

comment:21 Changed 14 months ago by zzz

Folders, drafts, and background sending in c4470e970c99ec248bb799a1108611e7862e1134 0.9.34-2 as described in comments 6 and 18 above.

Use with caution. More to follow in coming days. Cleanups and CSS still todo, esp. for buttons on show-mail page. Pushed early in the .35 cycle, even though not polished, so we get max test time. Issues/comments preferred here rather than a new ticket. Thanks.

comment:22 Changed 13 months ago by Reportage

Some general observations on the folder implementation (0.9.34-4):

  • When canceling a new mail, user should be returned to the folder/page they were viewing. Currently when a draft is canceled, user is taken to drafts folder.
  • Number of messages contained per folder should be indicated on the change folder dropdown (as an interim measure). A better implementation would be to display icon + folder name (for each folder) in a separate div in the header, with the number of messages displayed per folder, which would avoid the less than optimal dropdown and enable single click to change folder (and drag and drop message move?).
  • The trash folder is currently redundant, presumably that's a work in progress?

comment:23 Changed 13 months ago by zzz

Thanks for testing.
Sorry didn't leave a good list here of what was still to be done, partly because I didn't have a list myself.

  • cancel back-to-folder: agreed
  • Agreed about number of messages, been thinking about best way
  • Not sure yet about trash vs. delete
  • BCC to self vs. how sent folder works
  • Reply/replyall/forward doesn't fill in the compose form correctly (mentioned on zzz.i2p)

I did work on the CSS for the new buttons yesterday, looks a little better, still some issues.

Lots going on but we still have 8 weeks or so, will post updates here as I get to it.

comment:24 Changed 13 months ago by zzz

or, maybe reply/replyall/forward does work. But formatting of to: line is wacky. And forwarding an email with attachments loses the attachments. (I believe the attachment problem is also a bug in the .34-0 release and possibly even earlier)

comment:25 Changed 13 months ago by zzz

Reply/replyall/forward fixed in e3aeba35e7f26c9c364e1bde15926e213bac02a8 0.9.34-5

comment:26 Changed 13 months ago by zzz

Removed bcc-to-self feature, added copy-to-sent config option default true
in a37105c1984ead3a0aaad52672b9bd1c836a6035 to be 0.9.34-6

comment:27 Changed 13 months ago by Reportage

Moving pre-existing bcc-to-self e-mails from inbox to sent causes the mails to be redownloaded from the server.

As regards pre-existing bcc-to-self e-mails, is it worth implementing a batch-move script (perhaps accessible from the settings page) to automatically detect and move pre-existing sent mails to the sent mail folder (by correlating to:, from: and logged in account)?

comment:28 Changed 13 months ago by zzz

re: redownloading, can't reproduce here
re: batch move, might be nice to have, but can't immediately think of a clean way to do it in the UI

Add including of attachments when forwarding
in e2b980ab057561d5b60da27fec1fafba568eeea1 0.9.34-6

comment:29 Changed 13 months ago by Reportage

In order to reproduce the redownloaded bcc'd mails bug, the following conditions need to be met:

  • susimail is configured to retain mail on server
  • bcc-to-self e-mails need to exist on the server
  • a bcc-to-self e-mail is moved from inbox to sent

The problem appears to exist because when susimail checks for new mails, it's not looking in the sent folder for evidence of mails that exist on the server. Obviously, going forward this isn't likely to be an issue as the default behavior has changed to copy sent mail to the sent folder rather than send via the server, although if a mail is sent by a user with his own address manually configured for bcc (unlikely) this will trigger the issue.

So to verify the bug, manually add your own address to the bcc: field and then move the resulting mail to the sent folder (ensuring mail is being retained on the server) and then check for new mail.

comment:30 Changed 13 months ago by zzz

ok thanks for the details, should be able to figure it out from there

comment:31 Changed 13 months ago by Reportage

Mail folders implementation is clumsy. Fixes to that and other UX enhancements below:


I have attached the two diffs as files. Thank you, Reportage. - slumlord

Last edited 13 months ago by slumlord (previous) (diff)

comment:32 Changed 13 months ago by slumlord

Hi Reportage, it would be better if you could attach large diffs as files instead of pasting the whole diff here.

Changed 13 months ago by slumlord

Patch by Reportage - 1

Changed 13 months ago by slumlord

Patch by Reportage - 2

comment:33 Changed 12 months ago by zzz

re: comments 27-29, was unrelated to BCC-to-self. Delayed deletions weren't sent when checking mail, causing them to be re-downloaded if mail was explicitly checked before the delayed-deletion task ran.
Also fixed bug in fetching mails after check, CSS tweaks, and return to inbox after sending email from drafts
In d763a0c0858d8a7e4fe55c074cd5fe1cf66ea9d6 0.9.34-15
more to follow

comment:34 Changed 2 months ago by zzz

Add a subticket #2459.

Note: See TracTickets for help on using tickets.