Add DottedKey to a super table gives wrong output

I am trying to set dotted keys on an already existing super table. This seems to go wrong during the render method, because the code thinks I want to render everything as a super table.

My code:

import tomlkit

doc = tomlkit.loads("""
[output.netcdf_scalar]
value = 5
""")

output_table = doc["output"]
output_table[tomlkit.key(["csv", "path"])] = "output.csv"

print(tomlkit.dumps(doc))

Output:

csv.path = "output.csv"

[output.netcdf_scalar]
value = 5

Expected:

[output.csv]
path = "output.csv"

[output.netcdf_scalar]
value = 5

In the case of rendering the toml, it will say that the item "csv" is dotted, while "output" is not dotted, and "csv" is a super table, so it will not render "output" as prefix. But just adding the prefix will be wrong, because you actually need to at the [output.csv] table as a head, instead of writing output.csv.path.

tomlkit/tomlkit/container.py

Lines 551 to 566 in 8c963db

    
           for k, v in table.value.body: 
        
               if isinstance(v, Table): 
        
                   if ( 
        
                       cur.strip(" ") 
        
                       and not cur.strip(" ").endswith("\n") 
        
                       and "\n" not in v.trivia.indent 
        
                   ): 
        
                       cur += "\n" 
        
                   if v.is_super_table(): 
        
                       if k.is_dotted() and not key.is_dotted(): 
        
                           # Dotted key inside table 
        
                           cur += self._render_table(k, v) 
        
                       else: 
        
                           cur += self._render_table(k, v, prefix=_key) 
        
                   else: 
        
                       cur += self._render_table(k, v, prefix=_key)

I think that the _handle_dotted_key function is in this case not able to indicate that "csv" should not be super:

tomlkit/tomlkit/container.py

Lines 123 to 126 in 8c963db

    
           table = current = Table(Container(True), Trivia(), False, is_super_table=True) 
        
           for _name in mid: 
        
               _name._dotted = True 
        
               new_table = Table(Container(True), Trivia(), False, is_super_table=True)

No, the expected output should be

[output]
csv.path = "output.csv"

[output.netcdf_scalar]
value = 5

Furthermore, I likely won't have spare time on this, feel free to dig and contribute a fix.

Do you think it would be best to make a more automatic function for is_super_table() and remove the whole hardcoded setting of _is_super_table?

The problem is that output itself is now set as _is_super_table=True in the parser, but when adding extra keys to output, _is_super_table should be reevaluated.

Or would it be better update the renderer? is_super_table() for output indeed returns True, but there are already some extra checks in the renderer:

tomlkit/tomlkit/container.py

Lines 513 to 533 in 8c963db

    
           def _render_table(self, key: Key, table: Table, prefix: str | None = None) -> str: 
        
               cur = "" 
        
               if table.display_name is not None: 
        
                   _key = table.display_name 
        
               else: 
        
                   _key = key.as_string() 
        
                   if prefix is not None: 
        
                       _key = prefix + "." + _key 
        
               if not table.is_super_table() or ( 
        
                   any( 
        
                       not isinstance(v, (Table, AoT, Whitespace, Null)) 
        
                       for _, v in table.value.body 
        
                   ) 
        
                   and not key.is_dotted() 
        
               ): 
        
                   open_, close = "[", "]" 
        
                   if table.is_aot_element(): 
        
                       open_, close = "[[", "]]"

It would mean that we have to revise the check:

     if not table.is_super_table() or ( 
         any( 
             not isinstance(v, (Table, AoT, Whitespace, Null)) 
             for _, v in table.value.body 
         ) 
         and not key.is_dotted() 
     ):

In this case table.is_super_table() returns True, so it continues to the or statement. All instances are tables, and the output key itself is not dotted. So it will skip rendering the header.
Although output is a super table, we still want it to render, because... yeah, why actually? Because one of its keys is dotted? We see a similar if-statement in append():

tomlkit/tomlkit/container.py

Lines 278 to 288 in 8c963db

    
           is_table = isinstance(item, (Table, AoT)) 
        
           if ( 
        
               key is not None 
        
               and self._body 
        
               and not self._parsed 
        
               and (not is_table or key.is_dotted()) 
        
           ): 
        
               # If there is already at least one table in the current container 
        
               # and the given item is not a table, we need to find the last 
        
               # item that is not a table and insert after it 
        
               # If no such item exists, insert at the top of the table

As a very simple test I added a test that already contains an [output] table with a direct value and that works as expected:

def test_appending_to_super_table():
    content = """\
# Added these two lines
[output]
stuff = 3

[output.netcdf_scalar]
value = 5
"""

    doc = parse(content)
    output_table = doc["output"]
    output_table.append(tomlkit.key(["csv", "path"]), "output.csv")

    expected = """\
[output]
stuff = 3
csv.path = "output.csv"

[output.netcdf_scalar]
value = 5
"""

    assert doc.as_string() == expected

This is due to the fact that container:524 sees that output is indeed not a super table. But leaving out [output] stuff = 3 makes the test fail, because it thinks output is a super table. Then it would check the types inside of the table with

any(
    not isinstance(v, (Table, AoT, Whitespace, Null))
    for _, v in table.value.body
)
and not key.is_dotted()

But I guess that's not enough. What is the specific case I am looking for? Add an extra check for tables that contain dotted keys?

fix: Add DottedKey to a super table gives wrong output (#431) (#435)

Navigation Menu

Repository navigation

Add DottedKey to a super table gives wrong output #431

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	for k, v in table.value.body:
	if isinstance(v, Table):
	if (
	cur.strip(" ")
	and not cur.strip(" ").endswith("\n")
	and "\n" not in v.trivia.indent
	):
	cur += "\n"
	if v.is_super_table():
	if k.is_dotted() and not key.is_dotted():
	# Dotted key inside table
	cur += self._render_table(k, v)
	else:
	cur += self._render_table(k, v, prefix=_key)
	else:
	cur += self._render_table(k, v, prefix=_key)

	table = current = Table(Container(True), Trivia(), False, is_super_table=True)
	for _name in mid:
	_name._dotted = True
	new_table = Table(Container(True), Trivia(), False, is_super_table=True)

	def _render_table(self, key: Key, table: Table, prefix: str \| None = None) -> str:
	cur = ""

	if table.display_name is not None:
	_key = table.display_name
	else:
	_key = key.as_string()

	if prefix is not None:
	_key = prefix + "." + _key

	if not table.is_super_table() or (
	any(
	not isinstance(v, (Table, AoT, Whitespace, Null))
	for _, v in table.value.body
	)
	and not key.is_dotted()
	):
	open_, close = "[", "]"
	if table.is_aot_element():
	open_, close = "[[", "]]"

	is_table = isinstance(item, (Table, AoT))
	if (
	key is not None
	and self._body
	and not self._parsed
	and (not is_table or key.is_dotted())
	):
	# If there is already at least one table in the current container
	# and the given item is not a table, we need to find the last
	# item that is not a table and insert after it
	# If no such item exists, insert at the top of the table

Add DottedKey to a super table gives wrong output #431

Description

Activity

frostming commented on Jun 27, 2025

deltamarnix commented on Jun 27, 2025

deltamarnix commented on Jul 6, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions