#157 ✓closed
John Wright

Message tap pattern for AMQP transport

Reported by John Wright | January 18th, 2011 @ 06:25 PM | in 0.9.3

Currently the AMQP support is limited to direct queues. It'd be useful to extend this to be able to bind to other types of exchange in order to intercept messages. This will allow applications with their existing message flow to be integrated with Diffa via the changes frontend, without them having to change their messaging behaviour.

The AMQP portion of the changes frontend needs to be able to receive an existing application's multicast messages and convert them to Diffa's change event format in a pluggable way.

Comments and changes to this ticket

  • Ben Hood

    Ben Hood January 19th, 2011 @ 01:30 PM

    • Milestone cleared.
    • Milestone order changed from “36” to “0”
  • Ben Hood

    Ben Hood January 19th, 2011 @ 01:31 PM

    • Tag changed from “amqp json pluggable tap” to “amqp, json, pluggable, tap”
  • John Wright

    John Wright January 20th, 2011 @ 04:57 PM

    • Milestone cleared.
    • Tag changed from “amqp, json, pluggable, tap” to “amqp json pluggable tap”
    • Assigned user set to “John Wright”
  • Paul Jones

    Paul Jones January 21st, 2011 @ 11:38 AM

    • Milestone cleared.
  • Paul Jones

    Paul Jones January 21st, 2011 @ 11:39 AM

    • Tag changed from “amqp json pluggable tap” to “pluggable tap, amqp, json”
  • Ben Hood

    Ben Hood January 24th, 2011 @ 09:22 AM

    • State changed from “new” to “open”
  • John Wright

    John Wright January 26th, 2011 @ 01:34 PM

    • State changed from “open” to “resolved”
    • Milestone cleared.
    • Assigned user changed from “John Wright” to “Ben Hood”

    Ready for QA, branch 157 @ git://github.com/johnwright/diffa.git

  • Paul Jones

    Paul Jones January 26th, 2011 @ 02:30 PM

    • Milestone cleared.
    • Milestone order changed from “11” to “0”
  • Ben Hood

    Ben Hood January 26th, 2011 @ 03:53 PM

    • State changed from “resolved” to “open”
    • Assigned user changed from “Ben Hood” to “John Wright”

    QA notes:

    • Compile error - is this something local to me?
    INFO] /home/ben/Workspace/diffa/amqp-changes-test/src/main/scala:-1: info: compiling
[INFO] Compiling 1 source files to /home/ben/Workspace/diffa/amqp-changes-test/target/classes at 1296056665228
[ERROR] /home/ben/Workspace/diffa/amqp-changes-test/src/main/scala/net/lshift/diffa/messaging/example/ExampleEventFormatMapper.scala:19: error: EventFormatMapper is not a member of net.lshift.diffa.kernel.participants
[INFO] import net.lshift.diffa.kernel.participants.EventFormatMapper
[INFO]        ^
[ERROR] /home/ben/Workspace/diffa/amqp-changes-test/src/main/scala/net/lshift/diffa/messaging/example/ExampleEventFormatMapper.scala:29: error: not found: type EventFormatMapper
[INFO] class ExampleEventFormatMapper extends EventFormatMapper {
[INFO]                                        ^
[ERROR] two errors found

    • AMQPProducer:

      • L22: Is the BasicProperties import necessary?

    • AMQPQueueUrl:

      • L43: What about the FIXME?

    • EventFormatMapper

    • The map definition is not documented - this seems like an important trait behavior; neither is the content val

    • ExampleEventFormatMapperIntegrationTest:

    • This is good, but can we poll and re-try in a loop instead of waiting n secs?

    • JsonAmqpMessagingRegistrar:

    • L63: Could this log statement be more informative?

    • The mvn project name for the Changes module is inconsistent:

    ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] JavaMail File Transport ............................... SUCCESS [0.439s]
[INFO] LShift Diffa Parent POM ............................... SUCCESS [0.043s]
[INFO] LShift Diffa Kernel ................................... SUCCESS [0.159s]
[INFO] LShift Diffa JSON Messaging Support ................... SUCCESS [0.003s]
[INFO] LShift Diffa Participant Support ...................... SUCCESS [0.040s]
[INFO] LShift Diffa AMQP Messaging Support ................... SUCCESS [0.004s]
[INFO] LShift Diffa Tools .................................... SUCCESS [0.003s]
[INFO] LShift Diffa docgen-maven-plugin Maven Mojo ........... SUCCESS [0.003s]
[INFO] LShift Diffa Agent .................................... SUCCESS [0.125s]
[INFO] LShift Diffa Web Participants ......................... SUCCESS [0.063s]
[INFO] LShift Diffa Dist Packaging ........................... SUCCESS [0.061s]
[INFO] LShift AMQP Changes Integration Test .................. SUCCESS [0.048s]
  • John Wright

    John Wright January 26th, 2011 @ 05:21 PM

    • State changed from “open” to “resolved”
    • Assigned user changed from “John Wright” to “Ben Hood”

    Pushed fixes for QA.

  • Ben Hood

    Ben Hood January 26th, 2011 @ 05:31 PM

    The patch is foine, but now I'm getting this error:

    [INFO] Webapp directory = /home/ben/Workspace/diffa/amqp-changes-test/src/main/webapp
[INFO] Starting jetty 7.1.6.v20100715 ...
2011-01-26 17:28:25.719:INFO::jetty-7.1.6.v20100715
2011-01-26 17:28:25.721:INFO::Adding overlay: jar:file:/home/ben/Workspace/diffa/agent/target/diffa-agent-0.9.2-SNAPSHOT.war!/
2011-01-26 17:28:25.721:WARN::Failed startup of context JettyWebAppContext@74de1777@74de1777/diffa-agent,file:/home/ben/Workspace/diffa/amqp-changes-test/src/main/webapp,file:/home/ben/Workspace/diffa/amqp-changes-test/src/main/webapp
java.lang.IllegalArgumentException: file:/home/ben/Workspace/diffa/amqp-changes-test/src/main/webapp is not an existing directory.
    at org.eclipse.jetty.util.resource.ResourceCollection.setResources(ResourceCollection.java:87)
    at org.mortbay.jetty.plugin.MavenWebInfConfiguration.preConfigure(MavenWebInfConfiguration.java:139)
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:378)
    at org.mortbay.jetty.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:114)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
    at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:162)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
    at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:165)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
    at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:92)
    at org.eclipse.jetty.server.Server.doStart(Server.java:242)
    at org.mortbay.jetty.plugin.JettyServer.doStart(JettyServer.java:67)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55)
    at org.mortbay.jetty.plugin.AbstractJettyMojo.startJetty(AbstractJettyMojo.java:437)
    at org.mortbay.jetty.plugin.AbstractJettyMojo.execute(AbstractJettyMojo.java:377)
    at org.mortbay.jetty.plugin.JettyRunMojo.execute(JettyRunMojo.java:577)
    at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
    at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
    at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
    at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
    at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
    at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
2011-01-26 17:28:25.723:INFO::Started SelectChannelConnector@0.0.0.0:19095
[INFO] Started Jetty Server
[INFO] [surefire:test {execution: surefire-itest}]
[INFO] Surefire report directory: /home/ben/Workspace/diffa/amqp-changes-test/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running net.lshift.diffa.messaging.example.itest.ExampleEventFormatMapperIntegrationTest
17:28:26.375 [main] AmqpConnectionChecker.scala:31 - Checking for AMQP connection
17:28:26.541 [main] AmqpConnectionChecker.scala:35 - AMQP server properties: {product=RabbitMQ, information=Licensed under the MPL.  See http://www.rabbitmq.com/, platform=Erlang/OTP, copyright=Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd., version=2.1.0}
17:28:26.551 [main] AbstractRestClient.scala:31 - Configured to initialize using the server URL (http://localhost:19095/diffa-agent) with a sub URL (rest/diffs/)
17:28:26.847 [main] AbstractRestClient.scala:31 - Configured to initialize using the server URL (http://localhost:19095/diffa-agent) with a sub URL (rest/config/)
17:28:27.206 [main] AbstractRestClient.scala:84 - 503
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.07 sec <<< FAILURE!

Results :

Tests in error: 
  integrationTest(net.lshift.diffa.messaging.example.itest.ExampleEventFormatMapperIntegrationTest)

    The surefire report says this:

    ests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.07 sec <<< FAILURE!
integrationTest(net.lshift.diffa.messaging.example.itest.ExampleEventFormatMapperIntegrationTest)  Time elapsed: 0.981 sec  <<< ERROR!
java.lang.RuntimeException: 503
    at net.lshift.diffa.messaging.json.AbstractRestClient.create(AbstractRestClient.scala:85)
    at net.lshift.diffa.tools.client.ConfigurationRestClient.declareGroup(ConfigurationRestClient.scala:31)
    at net.lshift.diffa.messaging.example.itest.ExampleEventFormatMapperIntegrationTest.integrationTest(ExampleEventFormatMapperIntegrationTest.scala:48)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
    at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
    at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
    at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
  • Ben Hood

    Ben Hood January 26th, 2011 @ 05:42 PM

  • Ben Hood

    Ben Hood February 3rd, 2011 @ 06:16 PM

    • State changed from “resolved” to “closed”
  • Ben Hood

    Ben Hood February 4th, 2011 @ 02:48 PM

    • Milestone set to “0.9.3”
    • Milestone order changed from “14” to “0”

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

A Real Time Differencing Tool.

Shared Ticket Bins

People watching this ticket

Tags

Referenced by

Pages