Lightweight But Robust ETL Framework

A client was in need of an ETL framework that was flexible, simple, fault tolerant, configurable, and had built-in logging and auditing capabilities. The “simple” attribute made it difficult to find a suitable framework for my client but after evaluating several good frameworks, I finally settled on this one by Chris Price.

I really appreciate that it allows for the modular package design that Stéphane Fréchette advocates without much repetition to satisfy the framework. Below is a presentation about the framework that I encourage others in my field to review – especially if building one from scratch or hiring someone who might build one from scratch.

There is a short video about this presentation and a full video hosted on Pragmatic Works.

Data Integrity is Human Integrity

As I was reading How Will You Measure Your Life by Chris Christensen, it occurred to me with greater clarity that data integrity is merely a reflection of human integrity. Nobody wants data to contain errors – and yet it happens all the time despite our best intentions.

In organizations with a lot of data, there are often many people involved in data integrity: from the users who provide information to the people who interpret the reports – and everyone in between. But ask any person in the chain of data if they have integrity and they will always answer in the affirmative. However, the data is a reflection of the truth about our own integrity and it is rarely (never) flawless.

What I’ve observed is that data integrity issues are rarely (if ever) a result of a deliberate decision to enter incorrect data. Instead, it is often the result of rushed, tired, or stressed employees who are pressured to finish something quickly. This is true not only for the data entry clerks but for the people building the software, the business systems, and the reports as well.

Marginal thinking is a principle that is taught in most finance and economics courses. It tells us that in evaluating alternative investments, we should ignore costs that have already incurred and instead, base decisions on the new costs and new revenues that each alternative entails.

When faced with intense pressure to finish a project on time, the marginal cost of skipping a step just this once always seems negligible. We say to ourselves, “look, I know that as a general rule, most people shouldn’t do this – but in this particular extenuating circumstance, just this once, it’s OK.” The price of doing something wrong just this once usually appears alluringly low.

road-to-poor-dataFor example, Sam is a cook who was told that the Prime Minister has entered the restaurant and needs to finish his meal in one hour. Sam works swiftly to prepare the meal and, just before the meal is served, receives a phone call from one of the suppliers informing him that there’s a small chance the lettuce might have been on a truck that is suspected to have been contaminated by a poisonous substance.

As soon as Sam gets off the phone, the waitress hounds him about the status of the meal. To make matters more difficult, Sam recalls a time when his peers have taken a shortcut and received praise from the head chef for delivering on time versus another time when his peers have been hung up by an issue and received rash treatment for causing a delay.

In extenuating circumstances, most of us have been lulled into the “just this once” mindset. We justify making a small decision to break our own personal rules just one time. In the moment, it never feels like a major decision since the marginal costs are almost always low. But since life is just one unending stream of extenuating circumstances, each of those decisions aggregate and can turn us into someone we never aspired to be.

If the cook had crossed the line just that one time, he would’ve continued crossing it over and over again in various different ways. “It’s easier to hold to your principles 100% of the time than it is to hold to them 98% of the time. The first step down the wrong path is taken with a small decision. You justify all the small decisions that lead up to the big one and then you get to the big one and it doesn’t seem so enormous anymore.” – Chris Christensen in How Will You Measure Your Life

Data integrity is in constant battle with time. What makes it difficult is that the cost of skipping a step often seems lower than the cost of missing a tight timeline. Nobody will appreciate the correctness of data and everyone will point fingers if a deliverable is late. By the time any mistakes are found, it might be many months (or years) later. Few stakeholders appreciate the time required to check and verify aggregations. It’s necessary but yields no immediately tangible value that is easily measured.

How Will You Measure Your Life?

A few months ago, Jeff Bezos, CEO of Amazon.com, recommended 3 books that every CEO and business owner should read. I didn’t know which one to read first so I started investigating the authors who wrote them. I discovered that one of the authors, Clayton M. Christensen, wrote another book called How Will You Measure Your Life? The back cover read as follows:

In 2010 world-renowned innovation expert Clayton M. Christensen gave a powerful speech to the Harvard Business School’s graduating class. Drawing upon his business research, he offered a series of guidelines for finding meaning and happiness in life. He used examples from his own experiences to explain how high achievers can all too often fall into traps that lead to unhappiness.

The speech was memorable not only because it was deeply revealing but also because it came at a time of intense personal reflection: Christensen had just overcome the same type of cancer that had taken his father’s life. As Christensen struggled with the disease, the question “How do you measure your life?” became more urgent and poignant, and he began to share his insights more widely with family, friends, and students.

In this groundbreaking book, Christensen puts forth a series of questions: How can I be sure that I’ll find satisfaction in my career? How can I be sure that my personal relationships become enduring sources of happiness? How can I avoid compromising my integrity and stay out of jail? Using lessons from some of the world’s greatest businesses, he provides incredible insights into these challenging questions.

How Will You Measure Your Life? is full of inspiration and wisdom, and will help students, mid-career professionals, and parents alike forge their own paths to fulfillment.

I started reading the book using Audible.com’s incredible service but had to take a break. By the time I hit chapter 7, I was exhausted. The content is extremely good and, as the publisher stated, it is full of inspiration and wisdom. I had wanted to compile a really awesome post that summarizes key aspect of the book but I will share a few notes I had written down instead:

  • You can neglect the relationship with your spouse, and on a day-to-day basis it doesn’t seem like things are deteriorating.
  • Ambitious people may believe that their family is deeply important to them, they actually allocate fewer and fewer resources to the things they would SAY matter most. Few people set out to do this. The decisions often seem tactical. Just small decisions they think won’t have a large impact. But as they keep allocating resources in this way, they’re implementing a strategy different than what they intend.
  • How to check if you’re implementing the strategy you intend? Watch where your resources flow. If it doesn’t support the strategy that you’ve decided upon, you run the risk of a serious problem.
  • If the decisions you make about where to spend your blood sweat and tears are not consistent with the person you aspire to be, you’ll never become that person.
  • High achievers often spend too much time considering who they want to be at work and far too little time about who they want to be at home.
  • Getting something wrong doesn’t mean you failed, it just means you learned what doesn’t work.
  • When it seems like relationships at home are going well, you’ll be lulled into believing you can put these relationships onto the back burner. That’s a mistake. By the time serious issues arise in those relationships, it is often too late to repair them.

Towards the beginning of the book, the author discusses the difference between correlation and causation. People often attempt to replicate success by replicating the actions of other people; however, that is a lot like trying to fly by replicating a bird. As we can see from the early flight videos, it is far better to understand what causes a bird to fly than to simply attempt to replicate it. It wasn’t until we had a better understanding of fluid dynamics that our strategies improved.

3 Types of Team Players

In Chris Hadfield’s book, An Astronaut’s Guide To Life On Earth, he identifies three types of team players. I’ve included them below and added my own thoughts into the mix.

The -1

The -1 people are the team players who distract the team from heading in the right direction. They are the team players who get in the way and hinder progress. Nobody sets out to be a -1 and nobody would willingly admit to being a -1. Also, people who are -1 team players often think of themselves as quite the opposite. This is why it is always important to listen to feedback and criticism from other team members objectively and with humility.

The 0

The 0 people are team members who neither add or subtract value in the team. They don’t bring the team closer to their goals and they don’t take the team away from their goals. Interestingly, Chris says that his main objective when entering a new team is be a 0. He gives himself time to learn what everyone’s strengths and weaknesses are on the team and understand the goals and challenges with absolute clarity and certainty before attempting to add value.

The +1

The +1 people are team members who add value to a team. They are the ones who help keep the team moving closer and closer towards their goals. The +1 people try to build up other team members because that is how the whole will succeed. They also know when to return back to being a 0 so that others can be better +1s. If a person stays a +1 for too long, they might slip into being a -1.

I thought it was an excellent book and Chris Hadfield’s story is certainly an inspiration.

Team with Leadership

Team with Leadership

A team might be showing signs of bad fruit (like bickering, gossip, slander, stress, burnout, etc) if members in the team are heading in conflicting directions. Times like that need a leader to better clarify the common goal and re-explain the direction (how to get from where they are to where they must be).

Trying to address the bad fruit might be a waste of effort if the issue is a lack of sunlight.

What does a radish tree look like?

Radish vs AppleI thought it would be fun to write a short story in the style of a parable:

A child is born and grows into a man without ever knowing about radishes or apples. He has no idea what they are or where they come from. One day, a stranger hands him a radish and tells him it’s an apple. With only a radish in his hand, will the man know what an apple tree looks like? Of course not! If he bites into the radish and it has a bitter taste, will he ever seek out and find an apple tree that grows radishes? Of course not! Another stranger comes along and hands the man an apple and tells him it’s an apple. With the memory of a radish fresh in his mind, will the man even take a second look at the apple?

Obviously, this is a story to illustrate the dilemma that Christianity faces in a world where anybody can easily claim they are Christian without being questioned or confronted. It doesn’t mean the apple tree isn’t worth finding.

Seeing the Forest

Bread in BasketAt some point in our lives (if we’re lucky), we’ll need to prepare for our own death. That will probably be a slow and disturbing process of selling, giving, discarding and getting ready to say goodbye to everything else. Absolutely everything.

Mark 8 tells of a story about the disciples crossing a lake realizing that they had forgotten to bring food. Jesus warns them about King Herod and the religious leaders but the disciples began to argue about forgetting their food. After Jesus reminds them about the miraculous provisions they’ve seen from him, he ends with a question on verse 21: “Don’t you understand yet?” Interestingly, he also started with the same question and added this question: “Are your hearts too hard to take it in?”

Richard Sterns, author of Unfinished, wrote that the expression “He can’t see the forest through the trees” used to describe a person who is so focused on the things right in front of him that he has lost a sense of the bigger picture. He goes on to say that if we are ever truly going to find purpose and meaning in our lives, we first have to rise above the trees to rediscover the forest. In other words, we have to understand what God is doing in the world and how we fit in.

Verse 21 is ringing in my ears today and I’m contemplating about whether seeing the bigger picture is more a matter of the heart than it is a matter of the mind. If so, what does that require?

Rewrite Canada’s Prostitution Laws

Trafficking and Sexual ExploitationSince the Supreme Court ruling on Dec 20, 2013, Canada has one year to rewrite the prostitution laws and has asked for public consultation – ending March 17, 2014.

A study found that 95% of prostitutes in Canada (90% worldwide) would like to exit the business if they could and 95% of prostitutes experienced sexual harassment which would be legally actionable in a different job setting. But because of criminal conviction against prostitutes, it has been difficult to do anything about these issues – and even more difficult for trafficked women who might be here illegally, don’t know the language or laws well, and/or are terrified of their captors.

How we restructure the legal system is very important if we want to correctly address the issues of prostitution and sex trafficking in Canada. We can’t simply decriminalize the industry since an unrestrained demand will yield more trafficked women and children.

Now is the time to write letters to your local MP and the Department of Justice encouraging them to adopt the Nordic Model for Canada.

The Nordic Model is a tried, tested, and true model that has proven to greatly reduce sex trafficking and prostitution. It does this by criminalizing brothels, pimps, and buyers of sex while, at the same time, making it easier for prostitutes to get help exiting the industry. For more detailed information, read The Tipping Point by Joy Smith, MP.

Below are some very practical things that you can do which WILL have an impact on what Canada will look like in the generations to come. Please don’t be silent. Every voice matters.

Please note that the Nordic model is also strongly supported across political, religious and women’s rights organizations including, but not limited to:

  • Canadian Association of Sexual Assault Centres
  • Native Women’s Association of Canada
  • Canadian Association of Elizabeth Fry Societies
  • Action Ontarienne Contre La Violence Faite Aux Femmes
  • La Concentration Des Luttes Contra L’Exploitation Sexuelle
  • Le Regroupement Quebecois Des Centres d’Aide et De Lutte Contre Les Agressions A Caractere Sexual
  • Vancouver Rape Relief Society
  • Evangelical Fellowship of Canada
  • Defend Dignity
  • [free-them]
  • Walk With Me
  • Aboriginal Women’s Action Network
  • Asian Women Coalition Ending Prostitution
  • South Asian Women Against Male Violence
  • London Abused Women’s Centre
  • Catholic Women’s League
  • Sex Trade 101

Here are my answers to the public consultation in case anyone needs help:

1. Do you think that purchasing sexual services from an adult should be a criminal offence? Should there be any exceptions? Please explain.
Comment: Purchasing sexual services should absolutely be a criminal offence. The demand for sexual services will continue to grow as more and more generations become addicted to online gonzo porn – an increasingly popular sexual appetite that cannot be satisfied within the confines of a healthy relationship.

2. Do you think that selling sexual services by an adult should be a criminal offence? Should there be any exceptions? Please explain.
Comment: No. Criminalizing the selling of sexual services will make it more difficult for sex trafficking victims from coming forward. It will also make it harder for them to exit the trade if they need help to do so.

3. If you support allowing the sale or purchase of sexual services, what limitations should there be, if any, on where or how this can be conducted? Please explain.
Comment: Pimps shouldn’t be allowed to profit from the avails of someone else. Procuring sex and brothels should be illegal.

4. Do you think that it should be a criminal offence for a person to benefit economically from the prostitution of an adult? Should there be any exceptions? Please explain.
Comment: Yes. Permitting economical gain encourages sex trafficking and other forms of sexual exploitation.

5. Are there any other comments you wish to offer to inform the Government’s response to the Bedford decision?
Comment: Please take seriously the voices of sex trafficking victims. Their voices are being silenced so the few who can speak should be taken seriously and contrasted carefully against the vast majority of people who don’t fully understand the seriousness of the problem.

6. Are you are writing on behalf of an organization? If so, please identify the organization and your title or role:
Comment: No.

Block porn at the ISP level

Porn is a major force that’s negatively impacting personal development, marriages, relationships, children, and society as a whole – both nationally and globally.

It’s commonly a factor behind cases of rape, sex abuse, child abduction, sexual coercion, sex trafficking, prostitution, sexual dysfunction, adultery, divorce, family dissidence, and even cyber bullying.

Popular opinion is that porn itself is harmless and yet we are seeing more stories like this appear in the news: 13-Year-Old Boy Pleads Guilty to Raping His Younger Sister After Seeing Porn on an X-box.

The Internet generation of the 90s has been escalating into obscene porn addiction at alarming rates and each new generation is entering this fast-track to neurologically addictive material faster and faster.

According to research, there is consistent and reliable evidence that watching porn is directly related to male sexual aggression against women and that one in three boys are heavy porn users.

“Being a man means recognizing sexual violence and being outraged by it and doing [your] part to stop it.” – Barack Obama

The more people become addicted at a young age, the more people will have unhealthy and uncontrollable appetites for sex, and the more common child pornography, prostitution, and sex slavery will be in the next few decades – putting each new generation at an increasingly greater risk of exploitation at a young age.

“There can be no keener revelation of a society’s soul than the way in which it treats its children.” – Nelson Mandela

Knowing that laws don’t change hearts, how can we convince people not to look at porn and instead choose self-control, restraint, and compassion? How?

In the meantime, how can we reduce the number of children in future generations from encountering porn at an early age and risk getting addicted to it? There’s certainly no silver bullet here, but blocking porn by default at the Internet Service Provider (ISP) level could help. Households could then opt-in for receiving porn.

If you think a default porn block could help reduce the number of out-of-control sex predators in future generations, please sign the petition and send it to Mrs. Joy Smith, Member of Parliament who will present it to the House of Commons. With such a controversial subject, every signature counts. Please share this with friends, family, and colleagues.

The petition can be mailed postage-free to the address at the bottom of the petition page.

Download Petition

To stay informed of the issue, monitor Stop Porn Culture and Porn Harms. Both resources have newsletter signups and Facebook pages. I also encourage everyone to read about project Groundswell.