Installation of `kinesis-mock` failed in LocalStack

When I was setting up Kinesis on my local machine using LocalStack, I experienced an errorInstallation of kinesis-mock failed when I tried to create a Kinesis data stream. I googled the issue and found that someone reported the same issue in Github (ref: https://github.com/localstack/localstack/issues/8782) with a reply from LocalStack dev team. The reply did solve the issue, but it was not stated directly. Therefore, this post is to explain how I solved this issue with the guidance of LocalStack.

Problem

This is my docker-compose.yml file before the fix.

version: "3.8"

services:
  localstack:
    image: localstack/localstack:2.2.0
    ports:
      - "4566:4566"            
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"

When I tried to create a new Kinesis stream from LocalStack, I got the following error:

kinesis-learning-localstack-1  | 2023-08-28T06:31:44.491  INFO --- [   asgi_gw_0] l.s.k.kinesis_mock_server  : Creating kinesis backend for account 000000000000
kinesis-learning-localstack-1  | 2023-08-28T06:31:44.532 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain: Installation of kinesis-mock failed.
kinesis-learning-localstack-1  | 2023-08-28T06:31:44.533  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS kinesis.CreateStream => 500 (InternalError)

If you enable the debug mode (by defining DEBUG=1 in the environment), you will get the error details as below:

kinesis-learning-localstack-1  | 2023-08-28T06:31:10.954 DEBUG --- [   asgi_gw_0] l.services.plugins         : checking service health kinesis:4566
kinesis-learning-localstack-1  | 2023-08-28T06:31:10.978  INFO --- [   asgi_gw_0] l.s.k.kinesis_mock_server  : Creating kinesis backend for account 000000000000
kinesis-learning-localstack-1  | 2023-08-28T06:31:10.978 DEBUG --- [   asgi_gw_0] localstack.packages.api    : Starting installation of kinesis-mock...
kinesis-learning-localstack-1  | 2023-08-28T06:31:11.152 ERROR --- [   asgi_gw_0] l.aws.handlers.logging     : exception during call chain
kinesis-learning-localstack-1  | Traceback (most recent call last):
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 714, in urlopen
kinesis-learning-localstack-1  |     httplib_response = self._make_request(
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 403, in _make_request
kinesis-learning-localstack-1  |     self._validate_conn(conn)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn
kinesis-learning-localstack-1  |     conn.connect()
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
kinesis-learning-localstack-1  |     self.sock = ssl_wrap_socket(
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
kinesis-learning-localstack-1  |     ssl_sock = _ssl_wrap_socket_impl(
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
kinesis-learning-localstack-1  |     return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
kinesis-learning-localstack-1  |   File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
kinesis-learning-localstack-1  |     return self.sslsocket_class._create(
kinesis-learning-localstack-1  |   File "/usr/local/lib/python3.10/ssl.py", line 1071, in _create
kinesis-learning-localstack-1  |     self.do_handshake()
kinesis-learning-localstack-1  |   File "/usr/local/lib/python3.10/ssl.py", line 1342, in do_handshake
kinesis-learning-localstack-1  |     self._sslobj.do_handshake()
kinesis-learning-localstack-1  | ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1007)
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | During handling of the above exception, another exception occurred:
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | Traceback (most recent call last):
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
kinesis-learning-localstack-1  |     resp = conn.urlopen(
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 798, in urlopen
kinesis-learning-localstack-1  |     retries = retries.increment(
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
kinesis-learning-localstack-1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
kinesis-learning-localstack-1  | urllib3.exceptions.MaxRetryError: MyHTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/etspaceman/kinesis-mock/releases/tags/0.3.10 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1007)')))
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | During handling of the above exception, another exception occurred:
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | Traceback (most recent call last):
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/api.py", line 90, in install
kinesis-learning-localstack-1  |     self._install(target)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/core.py", line 127, in _install
kinesis-learning-localstack-1  |     super()._install(target)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/core.py", line 53, in _install
kinesis-learning-localstack-1  |     download_url = self._get_download_url()
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/core.py", line 150, in _get_download_url
kinesis-learning-localstack-1  |     response = requests.get(self.github_tag_url, headers=headers)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
kinesis-learning-localstack-1  |     return request("get", url, params=params, **kwargs)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
kinesis-learning-localstack-1  |     return session.request(method=method, url=url, **kwargs)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
kinesis-learning-localstack-1  |     resp = self.send(prep, **send_kwargs)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
kinesis-learning-localstack-1  |     r = adapter.send(request, **kwargs)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
kinesis-learning-localstack-1  |     raise SSLError(e, request=request)
kinesis-learning-localstack-1  | requests.exceptions.SSLError: MyHTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/etspaceman/kinesis-mock/releases/tags/0.3.10 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1007)')))
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | The above exception was the direct cause of the following exception:
kinesis-learning-localstack-1  | 
kinesis-learning-localstack-1  | Traceback (most recent call last):
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle
kinesis-learning-localstack-1  |     handler(self, self.context, response)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/handlers/service.py", line 123, in __call__
kinesis-learning-localstack-1  |     handler(chain, context, response)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/handlers/service.py", line 93, in __call__
kinesis-learning-localstack-1  |     skeleton_response = self.skeleton.invoke(context)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/skeleton.py", line 154, in invoke
kinesis-learning-localstack-1  |     return self.dispatch_request(context, instance)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/skeleton.py", line 166, in dispatch_request
kinesis-learning-localstack-1  |     result = handler(context, instance) or {}
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/forwarder.py", line 67, in _call
kinesis-learning-localstack-1  |     return fallthrough_handler(context, req)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/forwarder.py", line 92, in _forward_request
kinesis-learning-localstack-1  |     return forward_request(context, forward_url_getter)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/forwarder.py", line 103, in forward_request
kinesis-learning-localstack-1  |     return dispatch_to_backend(context, _call_http_backend)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/forwarder.py", line 120, in dispatch_to_backend
kinesis-learning-localstack-1  |     http_response = http_request_dispatcher(context)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/aws/forwarder.py", line 101, in _call_http_backend
kinesis-learning-localstack-1  |     return forward(context.request, forward_url_getter())
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/services/kinesis/provider.py", line 77, in get_forward_url
kinesis-learning-localstack-1  |     server = self.server_manager.get_server_for_account(account_id)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/services/kinesis/kinesis_mock_server.py", line 128, in get_server_for_account
kinesis-learning-localstack-1  |     self._servers[account_id] = self._create_kinesis_mock_server(account_id)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/services/kinesis/kinesis_mock_server.py", line 151, in _create_kinesis_mock_server
kinesis-learning-localstack-1  |     kinesismock_package.install()
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/api.py", line 195, in install
kinesis-learning-localstack-1  |     self.get_installer(version).install(target)
kinesis-learning-localstack-1  |   File "/opt/code/localstack/localstack/packages/api.py", line 98, in install
kinesis-learning-localstack-1  |     raise PackageException(f"Installation of {self.name} failed.") from e
kinesis-learning-localstack-1  | localstack.packages.api.PackageException: Installation of kinesis-mock failed.

The root cause of this issue is that the program failed to verify TLS/SSL of https://api.github.com

Solution

As stated in the reply in the Github issue, this issue is related to CA cert. To resolve this issue, we need to include the custom CA cert and defined NODE_EXTRA_CA_CERTS , CURL_CA_BUNDLE , and REQUESTS_CA_BUNDLE . However, it doesn’t state how to obtain the custom CA cert and how to resolve this issue in docker-compose use case.

Obtaining custom CA cert

Custom CA cert that should be used is different from one to another due to different environment setup. But the easiest way to obtain the custom CA cert is to visit the website directly by using your favourite web browser and save the CA cert used to verify the domain. In my example, I’m using Chrome to visit https://api.github.com and get the CA cert.

Export the CA cert and save it in the same folder where your docker-compose.yml is located.

Using custom CA cert in LocalStack docker

After obtaining custom CA cert, you can now use it in LocalStack docker by updating docker-compose.yml as following

version: "3.8"

services:
  localstack:
    image: localstack/localstack:2.2.0
    ports:
      - "4566:4566"            
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock
      - NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
      - CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
      - REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./custom-ca.cer:/etc/ssl/certs/ca-certificates.crt"

Restart docker and try to create Kinesis stream again, and it should work now!

Cheers!