Man-hours for implementing the “simple” Email sending module in an application with a modular architecture

On php sending mail is implemented in one line of code! And on java-need 3 weeks ??! (from conversations with developers and managers)

The article is not about how to send mail to java. My goal is to show the complexities of modular development of large applications (for example, the development of ERP River).

So, the task: to implement the service of sending by email (war).

Stages of development:

Let's start with sending

If not Spring (for a small module it is not needed), connect apache commons-email

<dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-email</artifactId> <version>1.3</version> </dependency> 

and write

  public class MailSender { public static void sendMail { HtmlEmail email = ... ? email.send(); 

Allow, where to get mail server settings? I think that even the youngest developer will never come up with their hardcode, therefore:

Mail Server Configuration

I hope we have already done the general work of configuring the entire system, and we can only implement it for the mail module:

  ... <part key="email"> <entry key="hostName">smtp.gmail.com</entry> <entry key="user">sendmail@mycompany.ru</entry> <entry key="password">psw</entry> <entry key="smtpPort">465</entry> <entry key="useSSL">true</entry> <entry key="debug">false</entry> <entry key="charset">UTF-8</entry> <entry key="useTLS">false</entry> </part> 

  public class MailConfig { public static <T extends Email> T prepareEmail(T email) { email.setHostName(hostName); email.setSmtpPort(port); email.setSSL(useSSL); email.setTLS(useTLS); email.setDebug(debug); email.setAuthenticator(defaultAuthenticator); email.setCharset(charset); return email; } 

Service call

Yeah, we have a service, how do we want to call it?
Business wants integration over web services, and you also need to have a simple HTTP GET send (for example, call directly from the browser):

• HTTP GET sending:
  

    public class MailServlet extends CommonServlet { @Override protected void doProcess(HttpServletRequest request, HttpServletResponse response, Map<String, String> params) throws IOException, ServletException { String from = ConfigUtil.getProperty("from", params); ... MailSender.sendMail(from, to, cc, ..); 

  
  
• Implementing a veb service (JAX-WS) is more complicated:
  ')
  

    @WebService @SOAPBinding(style = Style.RPC) public interface MailService { @WebMethod public void sendMail( @WebParam(name = "from") String from, @WebParam(name = "to") String to, @WebService(endpointInterface = "mycompany.MailService") public class MailServiceImpl implements MailService { @Override public void sendMail(String from, String to, String cc, String subject, String body, String attachmentUrls) throws StateException { MailSender.sendMailAndRecordHistory(from, to, cc, subject, body, ..); } 

  
  and mailService.wsdl:
  

    <definitions .. targetNamespace="http://mail.mycompany.com/" name="MailServiceImplService"> <message name="sendMail"> <part name="from" type="xsd:string"/> ... <portType name="MailService"> <operation name="sendMail" parameterOrder="from to cc subject body attachmentUrls"> <input wsam:Action="http://mail.mycompany.com/MailService/sendMailRequest" message="tns:sendMail"/> ... <binding name="MailServiceImplPortBinding" type="tns:MailService"> <soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="rpc"/> <operation name="sendMail"> <soap:operation soapAction=""/> ... <service name="MailServiceImplService"> <port name="MailServiceImplPort" binding="tns:MailServiceImplPortBinding"> <soap:address location="http://mycompany:8080/mail/mailService"/> ... 

  
  Do not forget web.xml (Tomcat)
  

    <listener> <listener-class>com.sun.xml.ws.transport.http.servlet.WSServletContextListener</listener-class> </listener> <servlet> <servlet-name>mailService</servlet-name> <servlet-class>com.sun.xml.ws.transport.http.servlet.WSServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>mailService</servlet-name> <url-pattern>/mailService</url-pattern> </servlet-mapping> <servlet> <servlet-name>mailServlet</servlet-name> <servlet-class>com.mycompany.mail.MailServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> ... 

Highlighting mail-client

And now how can the neighboring module of our system quickly pull our service on the web service? The easiest way is to select the mail-client maven module, make our mail service dependent on it, and allow any module — our client to include (maven dependency) mail-client:

• Make a separate maven mail-client module and put mailService.wsdl and interface MailService into it
  

    <groupId>com.mycompany</groupId> <artifactId>mail-client</artifactId> <name>Mail Client</name> 

  
  
• In addition, for the complete joy of our internal client, we make MailWSClient:
  calling the next module will be quite simple:
  

   MailWSClient.sendMail(... 

  
  

    public class MailWSClient { static String mailWsdl; private static final Service SERVICE; static { URL url = MailWSClient.class.getClassLoader().getResource("mailService.wsdl"); SERVICE = Service.create(url, new QName("http://mail.mycompany.com/", "MailServiceImplService")); // get mail endpoint from config mailWsdl = Config.getUrlAsString("mail/mailService?wsdl"); } public static void sendMail(String from, String to, ..){ getPort().sendMail(from, .. private static MailService getPort() { MailService port = SERVICE.getPort(MailService.class); Map<String, Object> requestContext = ((BindingProvider) port).getRequestContext(); requestContext.put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY, mailWsdl); return port; } 

We fasten patterns

Ege. In the workflow module we have 52 types of documents. It would be good for our customers to give the opportunity to determine the letter pattern themselves. Moreover, such a service (TemplateService) has already been implemented.
The template service is simple: it is implemented on jsp, the key and parameters are sent to get to it, the finished text is returned.

• Add sendTemplateMail to MailService, MailWSClient, MailServiceImpl, MailSender, mailService.wsdl and MailServlet:
  

   sendTemplateMail(.., templateKey, params); 

• And we implement it in MailSender (we already have a convenient wrapper MyHttpConnection, implemented through HttpURLConnection.openConnection ())
  

    static void sendTemplateMail(..., String key, String params) { LOGGER.info("Send template mail from ... String templateUrl = getUrlAsString("template?type=mail&format=html&key=" + key ... MyHttpConnection conn = MyHttpConnection.connect(templateUrl, params); if (conn.isOk()) { String body = conn.getMsg(); sendMail(from, to, cc, MailUtil.getSubject(body), body); } else { throw LOGGER.getStateException(conn.toString(), ExceptionType.TEMPLATE); ... 

  
  Along the way, I had to solve a problem with subject: the template service returns only the body of the letter. The template is returned in html format, MailUtil extracts tiltle from the template and uses it as subject:
  
  

    public class MailUtil { static Pattern MAIL_TITLE = Pattern.compile("<title>(.+)</title>", Pattern.MULTILINE); static String getSubject(String template) { Matcher m = MAIL_TITLE.matcher(template); return m.find() ? m.group(1) : null; } 

We send the document

Actually, we have documents. But what if we call the service with the document id? Templates for documents in the TemplateService already exist.

• Add sendDocMail to MailService, MailWSClient, MailServiceImpl, MailSender, mailService.wsdl and MailServlet.
  

   sendDocMail(String from, String to, String cc, String key, long docId); 

• Opanki, and the documents have attachments that need to be attached to the letter.
  Fortunately, commons-email allows for this easily, and we have a common maven attach-common module, which can ask for a list of attachments by docId:
  

    public class MailSender { static void sendDocMail(String from, String to, String cc, String key, long docId) throws StateException { List<Attach> list = AttachUtil.getList(docId); MailSender.sendTemplateMailAndRecordHistory(from, to, cc, key, "objectid=" + docId, MailUtil.formatAttach(list)); } public class MailUtil { // format attaches as // ulr1[name1], ulr2[name2], ... static String formatAttach(List<Attach> list) { return Util.collectionToDelimitedString(list, new Presentable<Attach>() { @Override public String toString(Attach attach) { return AttachConfig.downloadUrl + attach.getUuid() + '[' + attach.getName() + ']'; } 

fault tolerance

And if the server is temporarily unavailable? It is necessary to save the history in the database and do the add-on ... At the same time, we will solve the problem of sending a letter to the user for the purpose of the BPM task - it can be implemented via a trigger in the database: insert the TODO line into the table. As a side effect, we have a history of sending our messages, then you can wrap ui from above, and just do SQL table queries.
It’s good that we already have a scanning mechanism - just another implementation is needed.

• Making a mail_action table in the database
  

    CREATE TABLE hist.mail_action ( id SERIAL, _from TEXT, _to TEXT NOT NULL, _cc TEXT, subject TEXT, body TEXT, attachmenturls TEXT, state TEXT NOT NULL, date TIMESTAMP(0) WITHOUT TIME ZONE, key reference.ui_key, params TEXT ); 

• Add scan intervals to configuration
  

    <entry key="scanTodoInterval">30</entry> <entry key="scanFailInterval">600</entry> 

  
  

   scanTodoInterval = ConfigUtil.getInt(SCAN_TODO_INTERVAL, mailProps, 60); // default 60 sec scanFailInterval = ConfigUtil.getInt(SCAN_FAIL_INTERVAL, mailProps, 600); // default 10 min 

  
  We implement in MailSender a record of the sending history to the database along with the state (OK or Exception).
  We scan the mail_action table and send a message based on the state (TODO, EmailException)
  

    <listener> <listener-class>com.mycompany.common.web.SchedulerListener</listener-class> </listener> 

  
  

    public class MailWebScanner implements WebScheduler { private final MailScanner todoScanner = new MailScanner("TODO"); private final MailScanner failScanner = new MailScanner("org.apache.commons.mail.EmailException"); @Override public void activate(ServletContext servletContext) { todoScanner.startScanning(MailConfig.scanTodoInterval); failScanner.startScanning(MailConfig.scanFailInterval); } @Override public void deactivate() { todoScanner.deactivate(); failScanner.deactivate(); } @Override public void shutdown() { AsyncExecutor.shutdown(); } } public class MailScanner extends Scanner { private static final BeanListHandler<MailBean> HANDLER = new BeanListHandler<MailBean>(MailBean.class); private final String startWith; public MailScanner(String startWith) { this.startWith = startWith; } void startScanning(int interval) { activate(new Runnable() { @Override public void run() { for (MailBean mail : getMailToSend()) { MailSender.sendTemplateMailAndRecordHistory( } } }, interval, false); } ... List<MailBean> getMailToSend() { return SqlUtil.executeQuery("select * from hist.mail_action where state like '" + startWith + "%'", HANDLER); ... 

For those who do not like to wait: asynchrony

Since our service is now resilient to failures, we will allow our web service clients to not wait for a response. Instead of duplicating all the services of the service with the Async postfix and the @OneWay annotation, add the async flag and the AsyncExecutor call (our wrapper over ScheduledThreadPoolExecutor) to the MailWSClient calls:

  public class MailWSClient { public static void sendMail(final String from, final String to, final String cc, final String subject, final String body, final String attachmentUrls, boolean async) throws StateException { send(new Runnable() { @Override public void run() { getPort().sendMail(mask(from), mask(to), mask(cc), mask(subject), mask(body), mask(attachmentUrls)); } }, async); } public static void sendTemplateMail(final String from, final String to, final String cc, final String key, final String params, final String attachmentUrls, boolean async) throws StateException { ... public static void sendDocMail(final String from, final String to, final String cc, final String key, final long docId, boolean async) throws StateException { ... private static void send(Runnable task, boolean async) { if (async) { AsyncExecutor.submit(task); } else { task.run(); } } 

We fix image attachments

Well, it works! Finally, you can fix bugs - the pictures in the letter are not visible outside our intranet ... After all, they are set in our templates via <img src = “our internal resources”, naturally, you will not see them in the rest of the world.

Making them inline:

  public class MailSender { static void sendMailAndRecordHistory(String from, String to, String cc, String key, String params, String attachmentUrls, long docId) throws StateException { ... String embedImgBody = MailUtil.embedImg(body, email); public class MailUtil { static final Pattern HTML_URL = Pattern.compile("<img src=(?:\"|')(.+)(?:\"|')", Pattern.CASE_INSENSITIVE | Pattern.MULTILINE); public static String embedImg(String body, final HtmlEmail email) throws EmailException { return StringUtil.resolveReplacement(body, HTML_URL, new Presentable<Matcher>() { @Override public String toString(Matcher matcher) { String url = matcher.group(1); cid = email.embed(url, UUID.nameUUIDFromBytes(url.getBytes()).toString()); } return "<img src=\"cid:" + cid + "\""; ... 

We send embedded (data: image / png; base64, encoded_img) big pictures

A new business idea - by mistake from the client’s browser, send a screen screenshot to support mail.
The solution on UI is found - the move is ours. For the template service, write the error_mail.jsp template

  <%@page pageEncoding="UTF-8" %> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>Error Report</title> </head> <body> <h2>Error Report from '${user}'</h2> <b>Message:</b> <pre> ${message} </pre> <b>Screenshot:</b><br> <img src="${screenshot}"> </body> </html> 

Template parameters - exception message and base64_encoded_screenshot - are sent to the TemplateService from our service. We have problems: our self-written wrapper MyHttpConnection cannot send base64_encoded_screenshot via GET. You have to do POST and again do URLEncoder.encode because of problems with "+". In addition, in the inline mail that came in, the picture is not visible :( Well, you will also have to do it as an attachment:

  public class MailUtil { static Pattern DATA_PROTOCOL = Pattern.compile("^data:(.+);(.+),"); public static String embedImg(String body, final HtmlEmail email) throws EmailException { return StringUtil.resolveReplacement(body, HTML_URL, new Presentable<Matcher>() { @Override public String toString(Matcher matcher) { String url = matcher.group(1); String cid; try { Matcher m = DATA_PROTOCOL.matcher(url); if (m.find()) { final String cType = m.group(1); final String encoding = m.group(2); final String content = url.substring(m.toMatchResult().end()); cid = email.embed(new javax.activation.DataSource() { @Override public InputStream getInputStream() throws IOException { try { return javax.mail.internet.MimeUtility.decode(new ByteArrayInputStream(IOUtil.getBytes(content)), encoding); } catch (MessagingException e) { throw LOGGER.getIllegalStateException("Image encoding failed", e); } } // empty realization for other javax.activation.DataSource methods ... }, UUID.randomUUID().toString()); } else { cid = email.embed(url, UUID.nameUUIDFromBytes(url.getBytes()).toString()); } return "<img src=\"cid:" + cid + "\""; ... 

Final Point: Security

However, any user sends a get request from the browser - and receives an email with a completely secret document. Not good. It is necessary to fasten the access check of the user to the document with the transmitted docId and generally check: if the request came by get, if the user is logged into our system.
Due to the fact that the login page has already been done and there are a lot of things around it, and we have one entry point, I made a check through REST and domain level cookies with trust to server requests between the modules themselves, but this is already - separate article.

Summary of the simple task of sending mail:

The result was 2 maven modules with classes (not counting infrastructure such as configuration, attachments, templates, common parts and JUnit tests)

• mail-client
  
  • MailService: interface (sendMail, sendTemplateMail, sendDocMail)
  • MailWSClient: client-side wrapper exposing endPoint from configuration
  • mailService.wsdl
  
  
• mail-service
  
  • MailSender: actually sending
  • MailServiceImpl: web service implementation, delegation to MailSender
  • MailServlet: Servlet for handling HTTP GET
  • MailBean: bean to read a line from the database via commons-dbutils
  • MailConfig: configuration
  • MailScanner: table scan as sent
  • MailWebScanner: implementation of the listener for our service that launches 2 MailScanner scanners
  • MailUtil Utility Methods
  • EmailExceptionHandler: handling exceptions, AddressException fails
  • sun-jaxws.xml, web.xml
  
  

A patient reader who has come to the end of the article can himself compare the amount of labor costs for the “task of sending mail” and the resulting implementation. Thanks for attention.

---

References:

• Apache Email user guide
• Apache dbutils examples
• Screenshot generation
• ERP River