Preparing Seed Data for Your Application on Google App Engine

For the past few weeks, we have been getting requests on our website about the way we generated seed data for our multi-tenant application. I thought it would be best to talk about the method that we used as a part of a separate post.

If you have been following our blog, you would notice that TE-CON launched the public beta of bookmyhours.com. This is a an enterprise timesheet and invoicing application which is also multi-tenant. Thanks to the namespace api, we could convert the application to multi-tenant architecture in a matter of four hours. So how do we seed data for each tenant?

As soon as we get a request from a tenant like

amazon.bookmyhours.com/eh/login/Login/, the filter knows that the tenant is “amazon”. The namespace parameter is set for each request. Let us look at the filter logic

public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException,
			ServletException {

		// If the NamespaceManager state is already set up from the
		// context of the task creator the current namespace will not
		// be null. It's important to check that the current namespace
		// has not been set before setting it for this request.
		if (NamespaceManager.get() == null) {
			switch (strategy) {
			case SERVER_NAME: {
				String namespaceKey = request.getServerName();
				NamespaceManager.set(namespaceKey);
				logger.info("The namespace for the request is: " + namespaceKey);
				checkSeedDataAndNamespace(request, namespaceKey);
				break;
			}
			case GOOGLE_APPS_DOMAIN: {
				NamespaceManager.set(NamespaceManager.getGoogleAppsNamespace());
				break;
			}

			case EMPTY: {
				NamespaceManager.set("");
			}
			}
		}

		chain.doFilter(request, response);
	}

Once we get the tenant information from the request, we call a method called checkSeedDataAndNamespace(request, namespaceKey);

This method checks if the seed data is already present for this namespace. If it is not, then it is created else nothing is done.

Of course for added optimization, we check the memcache if the seed data exists instead of going to the datastore.

 

private void checkSeedDataAndNamespace(ServletRequest request, String namespaceKey) {
		HttpSession httpSession = ((HttpServletRequest) request).getSession();
		if (StringUtils.isEmpty((String) httpSession.getAttribute("isSeedDataSet"))) {
			if (CacheController.get(namespaceKey) == null) {
				seedDataSetup.setSeedData(false);
				CacheController.put(namespaceKey, "Seed Data Set");
			}
			httpSession.setAttribute("isSeedDataSet", "true");
		}
	}

Next, let us see how do we do the seeddata setup, just in case it does not exist

So, we have a method like

if (!dataExists) {
			setApplicationBootstrapData();
			adminAccountValidator.updateAdminPassword();
			if (runWithTestData){
				setBootstrapDataForTestingUser();
			}
		}

and the setApplicationBootstrapData() method looks like this,

private void setApplicationBootstrapData() {
		setConfigurationData();
		setMailTypeData();
		setprojectAssignmentType();
		setUserRoles();
		setUserDepartment();
		setUsers();

	}

As a sample, the setting up of user department happens like this

private void setUserDepartment() {
		UserDepartment department = new UserDepartment();
		department.setName("Internal");
		department.setCode("INT");
		userDepartmentDAO.persist(department);

		UserDepartment hrDepartment = new UserDepartment();
		hrDepartment.setName("Human Resources");
		hrDepartment.setCode("HR");
		userDepartmentDAO.persist(hrDepartment);

	}

Likewise we set up rest of the seed information which is required for a tenant. You would notice that since we are using JPA in our application, we call the persist on our GenericDAOJpaImpl class.

public T persist(T domObj) {
		entityManager.persist(domObj);
		return domObj;
	}

Hence, with the namespace api and the namespace filter, it becomes very easy to check if the seed data for a tenant is already set or it needs to be set for the first time.

You would like to keep some things in mind. There is a 30 second deadline at the app engine. If you feel that your seeding is going to take more than that then it is better to break the seeding into a few parts. First, set the part where the admin of the application is allowed to get in and is allowed to get to the admin screens. Once the admin is in the process of setting up the infrastructure you could seed other parts of the application as and when required.

Happy seeding!

Tagged: Cloud, GAE, Google App Engine, Initialization, Public Cloud, See Data

Posted in: Architecture, Cloud, Java