UsageReportingTask: SMTP with Authentication and STARTTLS

Installation & maintenance
1

Our SMTP server will no longer accept unauthorized access and requires STARTTLS. For the DataStoreServer I added the following entries in service.properties, which work perfect:

mail.smtp.starttls.enable = true
mail.smtp.starttls.required = true
mail.smtp.ssl.protocols = TLSv1.2
mail.smtp.auth = true
mail.smtp.host = ***.bam.de
mail.smtp.user = ***@bam.de
mail.smtp.password = ***
mail.from = ***@bam.de

For the UsageReportingTask I added the same lines to the service.properties of the ApplicationServer, but it doesnt work. When trying to send a mail an exception is thrown. Looks like it is not even trying to initialize the STARTTLS connection. Do I need to set other configuration values for the AS then for the DSS?

Thank you!

2023-04-25 08:35:39,771 INFO  [usage-reporting - Maintenance Plugin] OPERATION.MailClient - Sending message from '***@bam.de' to recipients '[***@bam.de]'
2023-04-25 08:35:39,795 ERROR [usage-reporting - Maintenance Plugin] OPERATION.MaintenancePlugin - Exception when running maintenance task 'ch.systemsx.cisd.openbis.generic.server.task.UsageReportingTask'.
ch.systemsx.cisd.common.exceptions.EnvironmentFailureException: Sending e-mail with subject 'Usage report for the period from 2023-04-16 until 2023-04-23' to recipients [***@bam.de] failed.
Detailed failure description:
javax.mail.AuthenticationFailedException: 334 NTLM supported

        at ch.systemsx.cisd.common.mail.MailClient.privateSendMessage(MailClient.java:440)
        at ch.systemsx.cisd.common.mail.MailClient.sendEmailMessageWithAttachment(MailClient.java:376)
        at ch.systemsx.cisd.openbis.generic.server.task.UsageReportingTask.sendReport(UsageReportingTask.java:220)
        at ch.systemsx.cisd.openbis.generic.server.task.UsageReportingTask.execute(UsageReportingTask.java:167)
        at ch.systemsx.cisd.common.maintenance.MaintenancePlugin$MaintenanceTimerTask.doRun(MaintenancePlugin.java:243)
        at ch.systemsx.cisd.common.maintenance.MaintenancePlugin$MaintenanceTimerTask.run(MaintenancePlugin.java:227)
        at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
        at java.base/java.util.TimerThread.run(Timer.java:506)
Caused by: javax.mail.AuthenticationFailedException: 334 NTLM supported

        at com.sun.mail.smtp.SMTPTransport$Authenticator.authenticate(SMTPTransport.java:648)
        at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:583)
        at javax.mail.Service.connect(Service.java:313)
        at javax.mail.Service.connect(Service.java:172)
        at javax.mail.Service.connect(Service.java:121)
        at javax.mail.Transport.send0(Transport.java:190)
        at javax.mail.Transport.send(Transport.java:120)
        at ch.systemsx.cisd.common.mail.MailClient.send(MailClient.java:455)
        at ch.systemsx.cisd.common.mail.MailClient.privateSendMessage(MailClient.java:415)
        ... 7 more
2

Dear @jraedler,
do you have it already solved?
Personally I would avoid using Java implementation of SMTP at all and our approach is to have system level Mail Transfer Agent on each instance (ususally Postfix or OpenSMTPD) with modern implementation of TLS. We configure openBIS to use that local MTA and then we relay emails through another SMTP relay or eventually allow direct email sending.

However if you did not solve it yet I can do investigation and try to find out how support of TLS 1.2 is supported. My first guess would be lack of support on Java level for this type of authentication.
I guess the mail.smtp.host = ***.bam.de is an Microsoft Exchange server. I often find tricky to use MS Exchange directly as SMTP from scripts or direct code in java or python.
I do not see the setting for port and at the moment I am not sure if we in openBIS code are using port 587 as the default port for SMTP submission or port 25 which is often not allowed anymore for submission with TLS enabled.

3

Dear Artur,

I gave up configuring openBIS to speak to our exchange server directly, the used smtp java implementation seems to be to old or to weak. Fun fact: DSS was working, AS was not, but both seem to use the same implementation.

Instead of using a full featured local MTA I just created a very small python script that acts as an SMTP relay on the openBIS server. It receives the messages on a local port without authentication, and sends it to our exchange server using TLS and credentials. The subject of the messages is changed to include an instance description and server name, so we can tell which of our openBIS instances was sending this mail.

If others have the same problem, feel free to use this code. It’s not tested very well, but works for us.

import asyncore, platform
from smtpd import SMTPServer
from smtplib import SMTP
from email import message_from_bytes

desc = 'main'

class RelayServer(SMTPServer):

    def process_message(self, peer, mFrom, mTo, mData, **kwargs):
        msg = message_from_bytes(mData)
        sbj = 'DataStore | %s (%s) | %s' % (desc, platform.node(), msg['Subject'])
        msg.replace_header('Subject', sbj)
        print('SMTP: %s > %s: %s' %(mFrom, mTo, sbj))
        s = SMTP('<smtp-server-name>')
        s.starttls()
        s.login('<username>', '<password>')
        s.send_message(msg)
        s.quit()

def run():
    RelayServer(('127.0.0.1', 1234), None)
    try:
        asyncore.loop()
    except KeyboardInterrupt:
        pass

if __name__ == '__main__':
    run()