Architectural review and refinement

This page includes the details of the post-experiment refined architecture.

Deployment View

Source UMLet diagram

Thunderforce lives inside Mozilla Thunderbird as an extension that communicates with the remote Salesforce.com servers using the simple object access protocol (SOAP) and caches information locally using SQLite. A SOAP implementation and SQLite are both built into the Mozilla platform.

Note that this approach will limit the number of Thunderforce clients that can run simultaneously due to API call limits. Thunderforce will include features that limit the number of API calls made in addition to basic API call statistics, such as expected versus actual API calls per hour and the total number of API calls over the past 24 hours. Those statistics can be divided into your organization's daily API call limits to determine how many Thunderforce installations you can have active during the day.

A future extension to the deployment view can involve an optional network server that receives workflow outbound messages from Salesforce.com and broadcasts those changes to local clients to reduce the amount of polling that Thunderforce has to do. Such a server can also substantially increase Thunderforce's scalability as it can be deployed to more computers without drastically increasing the number of API calls performed in a day. Perhaps a generic, client-agnostic solution similar to this might get built in the future (note that this is not a promise of future functionality).

Runtime View

Source UMLet diagram

Thunderforce is largely structured using the model-view-controller pattern to present Salesforce.com data to the end-user using bidirectional interconnects. With Salesforce.com as the master record, changes from any part of the system are pushed down to the lowest layer, and notifications regarding that change propagate from the lowest layer to the top, which is the view.

Salesforce.com's AJAX Toolkit and Mozilla Thunderbird's interfaces differ in various ways. To resolve this impedance mismatch, the Thunderbird interface implementation layer performs the necessary translations, implementing Salesforce.com as regular Thunderbird objects in the process.

Above the model layer, Thunderforce implements a minimal set of XML user interface language (XUL) overlays to extend the view and JavaScript-based user interface behavior override code to extend the controller. With this strategy, most of Thunderforce's functionality lives within the model and leverages Thunderbird's built-in functionality as much as possible.

Module View

Salesforce.com data layer (model)

Source UMLet diagram. This diagram will be revised soon to include LocalBidirectionalAPI and APIMetricsCollector.

The diagram above depicts how the Salesforce.com data layer components relate to each other.

In general, the methods in the API interface are those that are in the Salesforce.com AJAX toolkit. The AJAX toolkit's implementation will, indeed, exist mostly unchanged as the thunderforce.sforce.Salesforce object.

To turn Salesforce.com into a bidirectional API with event notification, the BidirectionalAPIPoller will wrap the Salesforce object to implement the BidirectionalAPI interface. This permits future extensibility with both non-polling solutions such as workflow outbound messages and aggregators such as local Salesforce.com cache servers.

To promote performance, the Cache object will store the returned values of method calls into memory and, with the offline functionality, a SQLite database. When notifications of changes from the backing store arrive, the cached return values that might be affected by those changes are invalidated. To balance performance with memory, the cache will manage its use of memory based on call and time statistics.

• As an example of cache invalidation, a change to an account object invalidates that object, that object's ancestor objects, queries that involve that entity, and searches that involve that entity
• Queries and searches are special functions that might be handled intelligently in specific update cases to further improve cache performance

In the cache, a write-back flag is included to effectively switch between online mode and offline mode. When write-back caching is enabled, the only the cache is updated. The backing store is not notified of the cached changes until the write-back flag is turned off. When the write-back flag is off, the caching becomes write-through, meaning that the backing store is notified synchronously. Note that event generation is performed by the cache when write-back is enabled. When write-back is disabled, the backing store generates the events that the cache propagates to its listeners. The cache will not, however, propagate events that do not alter the cached state, regardless of whether or not write-back is enabled.

This architectural pattern permits the upper layers to primarily interact with the BidirectionalAPI interface. As a result, the Cache class can be implemented in the future to get to the Dreamforce version as soon as possible. The LocalBidirectionalAPI will be implemented for Dreamforce as a naive bidirectional API that only generates events for local changes.

The BidirectionalAPIPoller presents a unidirectional API as a BidirectionalAPI through periodic polling for changes. To reduce the API call impact, only those entities that the user subscribed to within Thunderbird will be polled (question: will summary fields also cause the last-modified timestamp to change and trigger replication (WFOM and getUpdated)?). The set of subscribed entities is also what the Thunderforce searches are limited to, so we don't have to care about those entities that we are not subscribed to.

• Each entity will have its own polling frequency that can be set in setEntityRefreshIntervals(). For now, these will likely be set to default values internally. Eventually, this can be dynamically adjusted based on how frequently that entity is changing. An exponential back-off can be used when that entity has not changed since the last check.
• For subscribed entities that have the replicateable attribute, the replication methods getUpdated() and getDeleted() will be used to reduce the number of API calls to query different entities. If the user does not have permissions to use those methods, the poller will fall back to using entity queries
• If a large amount of data changed, the poller will notify listeners that possibly the entire state of the Salesforce.com data changed and that they should re-query anything that they are interested in. The Cache object will invalidate its entire cache upon receiving that event

The LocalBidirectionalAPI class simply generates events based on local changes, such as calls to update(), deleteIds(), etc. This will be implemented in the Dreamforce demo and can be used in the full version as an option to enhance local performance, though the Cache class will be needed to suppress duplicate events in the full version.

The APIMetricsCollector class collects API usage statistics and includes methods for getting the average number of API calls per hour, the total number of API calls performed in the previous 24 hours, the predicted amount of time left before API usage is exceeded, and other useful numbers. In the full version, a special window can show this information.

In the full implementation, the classes will likely be invoked with the following chain of delegates:

• Cache -> LocalBidirectionalAPI -> BidirectionalAPIPoller -> APIMetricsCollector -> Salesforce

In the Dreamforce implementation, the classes will likely be invoked with the following chain of delegates:

• LocalBidirectionalAPI -> Salesforce

Thunderbird interface implementation layer (model)

This section lists the Thunderforce classes and the interfaces that they implement.

• Account Type
  • nsThunderforceServer
    • @mozilla.org/messenger/server;1?type=imap
    • nsIMsgIncomingServer
    • nsISubscribableServer
  • nsThunderforceMessageService
    • @mozilla.org/messenger/messageservice;1?type=thunderforce
    • @mozilla.org/messenger/messageservice;1?type=thunderforce-message
    • @mozilla.org/messenger/protocol/info;1?type=thunderforce
    • @mozilla.org/network/protocol;1?name=thunderforce
    • @mozilla.org/uriloader/content-handler;1?type=x-application-thunderforcefolder
    • nsIMsgDBService
    • nsIMsgProtocolInfo
    • nsIProtocolHandler
    • nsIMsgMessageFetchPartService
    • nsIContentHandler
    • nsIMsgMessageService
  • nsThunderforceFolder
    • @mozilla.org/rdf/resource-factory;1?name=thunderforce
    • nsIMsgFolder
    • nsIMsgDatabase
    • nsIDBFolderInfo
    • nsICollection
    • nsICopyMessageListener
    • nsIClassInfo
    • nsIRDFResource
  • nsThunderforceMessageHeader
    • nsIMsgDBHdr
  • nsThunderforceAttachmentFolder
    • nsThunderforceFolder
  • nsThunderforceNoteFolder
    • nsThunderforceFolder
  • nsThunderforceEmailMessageFolder
    • nsThunderforceFolder
  • nsThunderforceDocumentFolder
    • nsThunderforceFolder
• Address Book
  • nsThunderforceAddressBook
    • nsIAbDirectory
    • nsIAbDirectoryQuery
    • nsIAbDirectorySearch
    • nsIClassInfo
    • nsIRDFResource
  • nsThunderforceAddressCard
    • nsIAbCard
  • nsThunderforceAddressGroup
    • nsIAbDirectory
• Message Composition
  • Implement nsISmtpService in nsThunderforceProtocol
  • Thunderforce-specific UI customizations can be handled via XUL and Javascript behavior extension
  • More research is required
• Message Filters
  • nsThunderforceMatchesSearchTerm
    • nsIMsgSearchTerm
  • nsThunderforceFilterListener
    • nsIMsgFilterHitNotify
  • nsThunderforceAssignmentRuleAction
    • nsIMsgRuleAction
  • XUL and Javascript behavior extensions can be used to inject the "Generate..." button, filter criteria additions, "matches" and "does not match" operators, filter values, filter actions, and filter action targets
  • More research is required
• Offline Mode
  • Handled by implementing the offline methods in nsThunderforceServer and nsThunderforceFolder

Folders

The following table lists which entity types as columns can be used to store an email message within an entity instance. Because the combination of Attachment and Document can be used on all entities, the Dreamforce release will turn all emails into either Attachment or Document objects when copying or moving a message to Salesforce.com.

Parent Entity	Attachment	Note	EmailMessage	Document
Account	yes	yes	no	no
Asset	yes	yes	no	no
Campaign	yes	no	no	no
Case	yes	no	yes	no
Contact	yes	yes	no	no
Contract	yes	yes	no	no
Custom Objects	yes	yes	no	no
EmailTemplate	yes	no	no	no
Event	yes	no	no	no
Folder	no	no	no	yes
Lead	yes	yes	no	no
Opportunity	yes	yes	no	no
Product2	yes	yes	no	no
Solution	yes	no	no	no
Task	yes	no	no	no
User	no	no	no	yes