The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

import-bot (20211) [Avatar] Offline
#1
Re: append to nested list ? (mutable object behavior)
[Originally posted by daryl harms]

Hi Bror,

Yes, you are running into one of the issues with mutable objects (i.e. list
and dictionaries) that catch just about everyone at least once!

What is happening here when you use the construct x = [[]] * 12 is that you
end up with a list that contains twelve references to the same empty list.
Rereading section 6.6 (pages 60-62) might help in picturing this. When you
then append to the first element (that is to the list referred to by the first
element) this is reflected in all the elements as they are referring to this
same list you just modified:

>>> dressMatrix = [[]] * 12
>>> dressMatrix
[[], [], [], [], [], [], [], [], [], [], [], []]
>>> dressMatrix[0].append(1)
>>> dressMatrix
[[1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]


To avoid this, you can explicitly type the empty lists in or set them in a
loop (so that you actually create 12 unique ones):

>>> dm2 =[[], [], [], [], [], [], [], [], [], [], [], []]
>>> dm2[0].append(1)
>>> dm2
[[1], [], [], [], [], [], [], [], [], [], [], []]
>>>
>>> dm3 =[]
>>> for i in range(12):
dm3.append([])

>>> dm3
[[], [], [], [], [], [], [], [], [], [], [], []]
>>> dm3[0].append(1)
>>> dm3
[[1], [], [], [], [], [], [], [], [], [], [], []]

Neither of these is unfortunately as concise as using the "*" operator. But
they more likely do what you want them to do.

Daryl
import-bot (20211) [Avatar] Offline
#2
[Originally posted by brjohan]

I've read QPB through twice and then some and learnt a lot (I think) about
Python. While experimenting with nested lists I've met a behaviour from Python
that I can't understand.

I want a 2-D matrix where each element is a list - initially empty. To this
empty list I want to append a number. Here follows what happens:

PythonWin 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
Portions Copyright 1994-2000 Mark Hammond (MHammond@skippinet.com.au)
>>> dressMatrix = [[]] * 12
>>> dressMatrix
[[], [], [], [], [], [], [], [], [], [], [], []]
>>> for i in range(len(dressMatrix)):
... dressMatrix[i] = [[]] * 12
...
>>> dressMatrix[0][0]
[]
>>> dressMatrix[0][0].append(1)
>>> dressMatrix[0]
[[1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]
>>> dressMatrix[:2]
[[[1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1]], [[], [], [],
[], [], [], [], [], [], [], [], []]]
>>>

Why does the 'append' method fill the complete row in the matrix with the
value that I supposed should be appended to the empty list referred to by
dressMatrix[0][0]?

Bror Johansson
import-bot (20211) [Avatar] Offline
#3
Re: append to nested list ? (mutable object behavior)
[Originally posted by brjohan]

Thank you Daryl,

I thought I was aware of that trap, but I obviously was not.

After some consideration I went for this construct:

dm = map(lambda x: [], range(12))

As the '12' in my example in reality should be a variable, I can't explicitly
line up a fixed number of empty lists to initialize the array.

Thank you for a good explanation - both here and in QPB.

/Bror

> Hi Bror,
>
> Yes, you are running into one of the issues with mutable objects (i.e. list
> and dictionaries) that catch just about everyone at least once!
>
> What is happening here when you use the construct x = [[]] * 12 is that you
> end up with a list that contains twelve references to the same empty list.
> Rereading section 6.6 (pages 60-62) might help in picturing this. When you
> then append to the first element (that is to the list referred to by the first
> element) this is reflected in all the elements as they are referring to this
> same list you just modified:
>
> >>> dressMatrix = [[]] * 12
> >>> dressMatrix
> [[], [], [], [], [], [], [], [], [], [], [], []]
> >>> dressMatrix[0].append(1)
> >>> dressMatrix
> [[1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]
>
>
> To avoid this, you can explicitly type the empty lists in or set them in a
> loop (so that you actually create 12 unique ones):
>
> >>> dm2 =[[], [], [], [], [], [], [], [], [], [], [], []]
> >>> dm2[0].append(1)
> >>> dm2
> [[1], [], [], [], [], [], [], [], [], [], [], []]
> >>>
> >>> dm3 =[]
> >>> for i in range(12):
> dm3.append([])
>
> >>> dm3
> [[], [], [], [], [], [], [], [], [], [], [], []]
> >>> dm3[0].append(1)
> >>> dm3
> [[1], [], [], [], [], [], [], [], [], [], [], []]
>
> Neither of these is unfortunately as concise as using the "*" operator. But
> they more likely do what you want them to do.
>
> Daryl
import-bot (20211) [Avatar] Offline
#4
Re: append to nested list ? (mutable object behavior)
[Originally posted by dale]

I ran into the same problem myself -- at about 3 AM and nowhere near an
internet connection-- so I came up with my own solution that I use to
initialize a rectangular "array" (AKA a list of lists):

height = 10
width = 20
row = [0] * width
array = [None] * height
for index in range(0, height):
array[index] = row[:]

I'll probably package it up as a function, say makeArray(height, width, value)
so I don't have to type the above every time.

My question is, is there a better way, or is there a pythonesque idiom for
doing this? My solution is a bit wordy.

PS Thanks for running this forum, Daryl. It's an incredible service!

Dale
import-bot (20211) [Avatar] Offline
#5
Re: append to nested list ? (mutable object behavior)
[Originally posted by daryl harms]

> I ran into the same problem myself -- at about 3 AM and nowhere near an
> internet connection-- so I came up with my own solution that I use to
> initialize a rectangular "array" (AKA a list of lists):
>
> height = 10
> width = 20
> row = [0] * width
> array = [None] * height
> for index in range(0, height):
> array[index] = row[:]
>
> I'll probably package it up as a function, say makeArray(height, width, value)
> so I don't have to type the above every time.
>
> My question is, is there a better way, or is there a pythonesque idiom for
> doing this? My solution is a bit wordy.
>
> PS Thanks for running this forum, Daryl. It's an incredible service!
>
> Dale

Hi Dale,

You have come up with almost exactly what I've seen the "experts" use in
Python. There are only a couple of very minor differences. You could use:
range(height)

rather than
range(0,height)

and wouldn't have to define the row on a separate line:

height = 10
width = 20
array = [None] * height
for index in range(height):
array[index] = [0] * width

Normally, though I like the idea of defining the row separately, as it makes
it a lot clearer what you are doing. Although in this case you then did have
to remember to make a copy of it (i.e. with row[:] ). So this is a toss up.

I like your idea of wrapping this in a function.

The only "Pythonesque" thing you might consider is whether a dictionary might
fit your needs in your specific situation better than a multi-dimensional
list. People coming from a language that doesn't have an easy to use built in
hash or dictionary like constructs to languages like Python or Java often
under use these constructs.

This is just something to consider as there are certainly many situations
where you really do want a multidimensional list.

Also if you are using these lists for matrices (and doing a lot of work with
them), you might want to take a look at Numeric Python which has a matrix
object type.

Take care (and be sure to get enough sleep smilie,
Daryl