logo
Splunk Team

using python mocking library on sqlalchemy

  • Last Update :
  • Techknowledgy :

First of all you can test session = DBSession() line also by

self.assertEqual(get_session('any_path'), mock_sessionmaker.return_value.return_value)

Moreover, mock_delarative_base.called is not an assert and cannot fail. Replace it with

self.assertTrue(mock_delarative_base.called)

Suggestion : 2

In these real-world examples, I will explain hypothetical or real scenarios in which I have used this library to mock SQLAlchemy in order to efficiently test my code. I will also explain several alternatives to this library to use for testing, and why specifically this library may be useful in the specific scenario.,In the real-world, a SQLAlchemy session can be interacted with multiple times to query some data. In those cases UnifiedAlchemyMagicMock can be used which combines various calls for easier assertions:,With the combination of this example and the previous example, we can use UnifiedAlchemyMagicMock to assert calls to check branching in code and verify data values using a mock SQLAlchemy session,Now, let us take a look at some example code for this scenario. First, let us consider the function we want to test. Please note the code below was created to support the scenario above and therefore is not runnable, but merely exemplary to what this library can perform.

$ pip install mock - alchemy
$ pip install "mock-alchemy>=0.1.0,<0.2.0"
>>> from mock_alchemy.comparison
import ExpressionMatcher
   >>>
   ExpressionMatcher(Model.foo == 5) == (Model.foo == 5)
True
>>> from mock_alchemy.mocking
import AlchemyMagicMock
   >>>
   session = AlchemyMagicMock() >>>
   session.query(Model).filter(Model.foo == 5).all()

   >>>
   session.query.return_value.filter.assert_called_once_with(Model.foo == 5)
>>> from mock_alchemy.mocking
import UnifiedAlchemyMagicMock
   >>>
   session = UnifiedAlchemyMagicMock()

   >>>
   m = session.query(Model) >>>
   q = m.filter(Model.foo == 5) >>>
   if condition:
   ...q = q.filter(Model.bar > 10).all() >>>
   data1 = q.all() >>>
   data2 = m.filter(Model.note == 'hello world').all()

   >>>
   session.filter.assert_has_calls([
      ...mock.call(Model.foo == 5, Model.bar > 10),
      ...mock.call(Model.note == 'hello world'),
      ...
   ])
>>> from mock_alchemy.mocking
import UnifiedAlchemyMagicMock
   >>>
   session = UnifiedAlchemyMagicMock(data = [
      ...(
         ...[mock.call.query(Model),
            ...mock.call.filter(Model.foo == 5, Model.bar > 10)
         ],
         ...[Model(foo = 5, bar = 11)]
         ...),
      ...(
         ...[mock.call.query(Model),
            ...mock.call.filter(Model.note == 'hello world')
         ],
         ...[Model(note = 'hello world')]
         ...),
      ...(
         ...[mock.call.query(AnotherModel),
            ...mock.call.filter(Model.foo == 5, Model.bar > 10)
         ],
         ...[AnotherModel(foo = 5, bar = 17)]
         ...),
      ...
   ]) >>>
   session.query(Model).filter(Model.foo == 5).filter(Model.bar > 10).all()[Model(foo = 5, bar = 11)] >>>
   session.query(Model).filter(Model.note == 'hello world').all()[Model(note = 'hello world')] >>>
   session.query(AnotherModel).filter(Model.foo == 5).filter(Model.bar > 10).all()[AnotherModel(foo = 5, bar = 17)] >>>
   session.query(AnotherModel).filter(Model.note == 'hello world').all()[]

Suggestion : 3

The Python Mock Library. The Python mock object library is unittest.mock. It provides an easy way to introduce mocks into your tests. Note: The standard library includes unittest.mock in Python 3.3 and later. If you’re using an older version of Python, you’ll need to install the official backport of the library. , 1 week ago The Python Mock Library. The Python mock object library is unittest.mock. It provides an easy way to introduce mocks into your tests. Note: The standard library includes unittest.mock in Python 3.3 and later. If you’re using an older version of Python, you’ll need to install the official backport of the library. ,I'm using sqlalchemy to query my databases for a project. Side-by-side, I'm new to unit testing and I'm trying to learn how can I make unit tests to test my database. I tried to use mocking library to test but so far, it seems to be very difficult., SQLAlchemy is awesome. Unittests are great. Accessing DB during tests - not so much. This library provides easy way to mock SQLAlchemy’s session in unittests while preserving ability to do sane asserts. Normally SQLAlchemy’s expressions cannot be easily compared as comparison on binary expression produces yet another binary expression: 


from sqlalchemy
import create_engine from sqlalchemy.orm
import sessionmaker from sqlalchemy.ext.declarative
import declarative_base from sqlalchemy.exc
import OperationalError, ArgumentError test_db_string = 'postgresql+psycopg2://testdb:[email protected]/'\
'test_databasetable'
def get_session(database_connection_string): try: Base = declarative_base() engine = create_engine(database_connection_string) Base.metadata.bind = engine DBSession = sessionmaker(bind = engine) session = DBSession() connection = session.connection() return session except OperationalError: return None except ArgumentError: return None

self.assertEqual(get_session('any_path'), mock_sessionmaker.return_value.return_value)
from sqlalchemy
import create_engine from sqlalchemy.orm
import sessionmaker from sqlalchemy.ext.declarative
import declarative_base from sqlalchemy.exc
import OperationalError, ArgumentError test_db_string = 'postgresql+psycopg2://testdb:[email protected]/'\
'test_databasetable'
def get_session(database_connection_string): try: Base = declarative_base() engine = create_engine(database_connection_string) Base.metadata.bind = engine DBSession = sessionmaker(bind = engine) session = DBSession() connection = session.connection() return sessionexcept OperationalError: return Noneexcept ArgumentError: return None
import mock
import unittest from mock
import sentinel
import get_session class TestUtilMock(unittest.TestCase): @mock.patch('app.Utilities.util.create_engine') # mention the whole path @mock.patch('app.Utilities.util.sessionmaker') @mock.patch('app.Utilities.util.declarative_base') def test_get_session1(self, mock_delarative_base, mock_sessionmaker, mock_create_engine): mock_create_engine.return_value = sentinel.engine get_session('any_path') mock_delarative_base.called mock_create_engine.assert_called_once_with('any_path') mock_sessionmaker.assert_called_once_with(bind = sentinel.engine)
self.assertEqual(get_session('any_path'), mock_sessionmaker.return_value.return_value)
self.assertTrue(mock_delarative_base.called)

Suggestion : 4

SQLAlchemy is awesome. Unittests are great. Accessing DB during tests - not so much. This library provides easy way to mock SQLAlchemy’s session in unittests while preserving ability to do sane asserts. Normally SQLAlchemy’s expressions cannot be easily compared as comparison on binary expression produces yet another binary expression:,You can install alchemy-mock using pip:,Alternatively AlchemyMagicMock can be used to mock out SQLAlchemy session:,In real world though session can be interacted with multiple times to query some data. In those cases UnifiedAlchemyMagicMock can be used which combines various calls for easier assertions:

You can install alchemy-mock using pip:

$ pip install alchemy - mock

SQLAlchemy is awesome. Unittests are great. Accessing DB during tests - not so much. This library provides easy way to mock SQLAlchemy’s session in unittests while preserving ability to do sane asserts. Normally SQLAlchemy’s expressions cannot be easily compared as comparison on binary expression produces yet another binary expression:

>>> type((Model.foo == 5) == (Model.bar == 5))
<class 'sqlalchemy.sql.elements.BinaryExpression'>

But they can be compared with this library:

>>> ExpressionMatcher(Model.foo == 5) == (Model.bar == 5)
False

ExpressionMatcher can be directly used:

>>> from alchemy_mock.comparison
import ExpressionMatcher
   >>>
   ExpressionMatcher(Model.foo == 5) == (Model.foo == 5)
True

Alternatively AlchemyMagicMock can be used to mock out SQLAlchemy session:

>>> from alchemy_mock.mocking
import AlchemyMagicMock
   >>>
   session = AlchemyMagicMock() >>>
   session.query(Model).filter(Model.foo == 5).all()

   >>>
   session.query.return_value.filter.assert_called_once_with(Model.foo == 5)

In real world though session can be interacted with multiple times to query some data. In those cases UnifiedAlchemyMagicMock can be used which combines various calls for easier assertions:

>>> from alchemy_mock.mocking
import UnifiedAlchemyMagicMock
   >>>
   session = UnifiedAlchemyMagicMock()

   >>>
   m = session.query(Model) >>>
   q = m.filter(Model.foo == 5) >>>
   if condition:
   ...q = q.filter(Model.bar > 10).all() >>>
   data1 = q.all() >>>
   data2 = m.filter(Model.note == 'hello world').all()

   >>>
   session.filter.assert_has_calls([
      ...mock.call(Model.foo == 5, Model.bar > 10),
      ...mock.call(Model.note == 'hello world'),
      ...
   ])

Finally UnifiedAlchemyMagicMock can partially fake session mutations such as session.add(instance). For example:

>>> session = UnifiedAlchemyMagicMock() >>>
   session.add(Model(pk = 1, foo = 'bar')) >>>
   session.add(Model(pk = 2, foo = 'baz')) >>>
   session.query(Model).all()[Model(foo = 'bar'), Model(foo = 'baz')] >>>
   session.query(Model).get(1)
Model(foo = 'bar') >>>
   session.query(Model).get(2)
Model(foo = 'baz')

Note that its partially correct since if added models are filtered on, session is unable to actually apply any filters so it returns everything:

>>> session.query(Model).filter(Model.foo == 'bar').all()[Model(foo = 'bar'), Model(foo = 'baz')]

Copyright (c) 2018, Miroslav Shubernetskiy

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files(the "Software"), to deal in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and / or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

   The above copyright notice and this permission notice shall be included in
   all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Suggestion : 5

SQLAlchemy is awesome. Unittests are great. Accessing DB during tests - not so much. This library provides easy way to mock SQLAlchemy's session in unittests while preserving ability to do sane asserts. Normally SQLAlchemy's expressions cannot be easily compared as comparison on binary expression produces yet another binary expression:,In real world though session can be interacted with multiple times to query some data. In those cases UnifiedAlchemyMagicMock can be used which combines various calls for easier assertions:,Alternatively AlchemyMagicMock can be used to mock out SQLAlchemy session:,Finally UnifiedAlchemyMagicMock can partially fake session mutations such as session.add(instance). For example:

You can install alchemy-mock using pip:

$ pip install alchemy - mock

SQLAlchemy is awesome. Unittests are great. Accessing DB during tests - not so much. This library provides easy way to mock SQLAlchemy's session in unittests while preserving ability to do sane asserts. Normally SQLAlchemy's expressions cannot be easily compared as comparison on binary expression produces yet another binary expression:

>>> type((Model.foo == 5) == (Model.bar == 5))
<class 'sqlalchemy.sql.elements.BinaryExpression'>

But they can be compared with this library:

>>> ExpressionMatcher(Model.foo == 5) == (Model.bar == 5)
False

Alternatively AlchemyMagicMock can be used to mock out SQLAlchemy session:

>>> from alchemy_mock.mocking
import AlchemyMagicMock
   >>>
   session = AlchemyMagicMock() >>>
   session.query(Model).filter(Model.foo == 5).all()

   >>>
   session.query.return_value.filter.assert_called_once_with(Model.foo == 5)

In real world though session can be interacted with multiple times to query some data. In those cases UnifiedAlchemyMagicMock can be used which combines various calls for easier assertions:

>>> from alchemy_mock.mocking
import UnifiedAlchemyMagicMock
   >>>
   session = UnifiedAlchemyMagicMock()

   >>>
   m = session.query(Model) >>>
   q = m.filter(Model.foo == 5) >>>
   if condition:
   ...q = q.filter(Model.bar > 10).all() >>>
   data1 = q.all() >>>
   data2 = m.filter(Model.note == 'hello world').all()

   >>>
   session.filter.assert_has_calls([
      ...mock.call(Model.foo == 5, Model.bar > 10),
      ...mock.call(Model.note == 'hello world'),
      ...
   ])

Suggestion : 6

In these real-world examples, I will explain hypothetical or real scenarios in which I have used this library to mock SQLAlchemy in order to efficiently test my code. I will also explain several alternatives to this library to use for testing, and why specifically this library may be useful in the specific scenario.,Now, let us take a look at some example code for this scenario. First, let us consider the function we want to test. Please note the code below was created to support the scenario above and therefore is not runnable, but merely exemplary to what this library can perform.,I will include some basic examples here to illustrate the use cases of this library. Feel free to check out the GitHub repository for the source code. ,When using the mock-alchemy package, the test function can now test this complex_data_analysis function despite it containing multiple calls to SQL and combining those calls. Here is an example of how this might look. Assume the file detailed above is called data_analysis.

from sqlalchemy
import Column, Integer, String
from sqlalchemy.ext.declarative
import declarative_base
Base = declarative_base()

# assume similar classes
for Data2 and Data3
class Data1(Base):
   __tablename__ = 'some_table'
pk1 = Column(Integer, primary_key = True)
data_val1 = Column(Integer)
data_val2 = Column(Integer)
data_val3 = Column(Integer)
def __init__(self, pk1, val1, val2, val3):
   self.pk1 = pk1
self.data_val1 = val1
self.data_val2 = val2
self.data_val3 = val3

class CombinedAnalysis(Base):
   __tablename__ = 'some_table'
pk1 = Column(Integer, primary_key = True)
analysis_val1 = Column(Integer)
analysis_val2 = Column(Integer)
analysis_val3 = Column(Integer)
def __init__(self, pk1, val1, val2, val3):
   self.pk1 = pk1
self.analysis_val1 = val1
self.analysis_val2 = val2
self.analysis_val3 = val3

def __eq__(self, other):
   if not isinstance(other, CombinedAnalysis):
   return NotImplemented
return (
   self.analysis_val1 == other.analysis_val1 and self.analysis_val2 == other.analysis_val2 and self.analysis_val3 == other.analysis_val3
)

def complex_data_analysis(cfg, session):
   # collects some data upto some point
dataset1 = session.query(Data1).filter(Data1.utc_time < cfg["final_time"])
dataset2 = session.query(Data2).filter(Data2.utc_time < cfg["final_time"])
dataset3 = session.query(Data3).filter(Data3.utc_time < cfg["final_time"])
# performs some analysis
analysis12 = analysis(dataset1, dataset2)
analysis13 = analysis(dataset1, dataset3)
analysis23 = analysis(dataset2, dataset3)
# combine the data analysis(returns object CombinedAnalysis)
combined_analysis = intergrate_analysis(analysis12, analysis13, analysis23)
# assume the combined_analysis are stored in some SQL table
self.session.add_all(combined_analysis)
self.session.commit()
import datetime
import mock

import pytest
from mock_alchemy.mocking
import UnifiedAlchemyMagicMock

from data_analysis
import complex_data_analysis, Data1, Data2, Data3, CombinedAnalysis

def test_data_analysis():
   stop_time = datetime.datetime.utcnow()
cfg = {
   "final_time": stop_time
}
data1_values = [
   Data1(1, some, data, values),
   Data1(2, some, data, values),
   Data1(3, some, data, values),
]
data2_values = [
   Data2(1, some, data, values),
   Data2(2, some, data, values),
   Data2(3, some, data, values),
]
data3_values = [
   Data3(1, some, data, values),
   Data3(2, some, data, values),
   Data3(3, some, data, values),
]
session = UnifiedAlchemyMagicMock(data = [
   (
      [mock.call.query(Data1),
         mock.call.filter(Data1.utc_time < stop_time)
      ],
      data1_values
   ),
   (
      [mock.call.query(Data2),
         mock.call.filter(Data2.utc_time < stop_time)
      ],
      data2_values
   ),
   (
      [mock.call.query(Data3),
         mock.call.filter(Data3.utc_time < stop_time)
      ],
      data3_values
   ),
])
complex_data_analysis(cfg, session)
expected_anyalsis = [
   CombinedAnalysis(1, some, anyalsis, values),
   CombinedAnalysis(2, some, anyalsis, values),
   CombinedAnalysis(3, some, anyalsis, values),
]
combined_anyalsis = session.query(CombinedAnalysis).all()
assert sorted(combined_anyalsis, key = lambda x: x.pk1) == sorted(expected_anyalsis, key = lambda x: x.pk1)

Suggestion : 7

Additionally, mock provides a patch() decorator that handles patching module and class level attributes within the scope of a test, along with sentinel for creating unique objects. See the quick guide for some examples of how to use Mock, MagicMock and patch().,If a mock instance with a name or a spec is assigned to an attribute it won’t be considered in the sealing chain. This allows one to prevent seal from fixing part of the mock object.,mock already provides a feature to help with this, called speccing. If you use a class or instance as the spec for a mock then you can only access attributes on the mock that exist on the real class:,The patch decorators are used for patching objects only within the scope of the function they decorate. They automatically handle the unpatching for you, even if exceptions are raised. All of these functions can also be used in with statements or as class decorators.

>>> from unittest.mock
import MagicMock
   >>>
   thing = ProductionClass() >>>
   thing.method = MagicMock(return_value = 3) >>>
   thing.method(3, 4, 5, key = 'value')
3
   >>>
   thing.method.assert_called_with(3, 4, 5, key = 'value')
>>> mock = Mock(side_effect = KeyError('foo')) >>>
   mock()
Traceback(most recent call last):
   ...
   KeyError: 'foo'
>>> values = {
      'a': 1,
      'b': 2,
      'c': 3
   } >>>
   def side_effect(arg):
   ...
   return values[arg]
      ...
      >>>
      mock.side_effect = side_effect >>>
      mock('a'), mock('b'), mock('c')
      (1, 2, 3) >>>
      mock.side_effect = [5, 4, 3, 2, 1] >>>
      mock(), mock(), mock()
      (5, 4, 3)
>>> from unittest.mock
import patch
   >>>
   @patch('module.ClassName2')
   ...@patch('module.ClassName1')
   ...def test(MockClass1, MockClass2):
   ...module.ClassName1()
   ...module.ClassName2()
   ...assert MockClass1 is module.ClassName1
   ...assert MockClass2 is module.ClassName2
   ...assert MockClass1.called
   ...assert MockClass2.called
   ...
   >>>
   test()
>>> with patch.object(ProductionClass, 'method', return_value = None) as mock_method:
   ...thing = ProductionClass()
   ...thing.method(1, 2, 3)
   ...
   >>>
   mock_method.assert_called_once_with(1, 2, 3)
>>> foo = {
      'key': 'value'
   } >>>
   original = foo.copy() >>>
   with patch.dict(foo, {
      'newkey': 'newvalue'
   }, clear = True):
   ...assert foo == {
      'newkey': 'newvalue'
   }
   ...
   >>>
   assert foo == original


Trending Technology


Most popular in python