Joseph Cheung (9) [Avatar] Offline
#1
Hi,

I got 2 questions after watching module 3:

1. Since we are running tests against real external resources e.g. DynamoDB / AWS Cognito, wouldn't it cost $ to run tests? I guess in this case we should not run integration tests and acceptance tests in watch mode (tests run every time when a file is changed), otherwise the bill may grow substantially just because of testing.

2. When we test `get-restaurants` and `search-restaurants` handler, we are asserting that the count of returned restaurants to be a certain number. I guess it's okay in the demo because it's just an illustration. However in real world, should we set up a separate test DynamoDB instance on AWS for this? Otherwise the test may fail if there's update to db items. And should we set up / tear down the test DynamoDB instance for every test case in order to make test result consistent?

Best,
Joseph
Yan Cui (68) [Avatar] Offline
#2
Hi Joseph,

1) yes, it would cost you to run integration in this case though the cost would be minimal. And yes, I would turn off watch mode for everything except for strictly unit tests that can be executed locally and quickly

2) my preference is to setup and tear down before and after the test, avoid using hard coded IDs, names etc. in the setup, and focus testing properties/behaviours (e.g. search-restaurants should only return restaurants with the theme I specified) and not specific result (i.e. when I search for theme cartoon, search-restaurants should return Lil' Bits); I don't think it's ever realistic to be using different tables for these tests, you're testing your code integrates with X, not something that's probably like X unless someone forgot to update it when I last made a change to X
Joseph Cheung (9) [Avatar] Offline
#3
Hi Yan,

Thank you for your answer!

I agree on your point of testing your code's integration with X and I believe using CI/CD in module 5 we can make sure the integration and acceptance tests are all green when a developer push his code while the cost running those tests are minimal since the tests are run on the cloud per change.

For point 2, using our demo app's `get-restaurants` test as an example to elaborate what I think:

1. We have a `restaurants-test` DynamoDB table on AWS
2. The test case has its own setup script to
  • clear up the existing `restaurants-test` table records

  • populate the table with 8 restaurants, which is the number of restaurants we expect from the API call in the testing code

  • 3. After asserting the test case, we have tear down script to clean up what the test may have altered the database

    I think step 3 is needed when we have `update-restaurants` / `remove-restaurants` handler which actually alter the test DynamoDB table on AWS. What do you think?

    Best,
    Joseph
    Yan Cui (68) [Avatar] Offline
    #4
    Hi Joseph,

    Sorry it took me a while to get back to you on this, totally got sidetracked, my bad! Jez asked a similar question which reminded me that you alreayd asked a similar question here.

    Take a look at my reply to Jez, I think it should answer your question here as well.
    Joseph Cheung (9) [Avatar] Offline
    #5
    Hi Yan,

    No worries! Just read your reply in the other thread which solves my question. Look forward to the upcoming videos!

    Best,
    Joseph