Example Error Message

Resolution

Modify the Query

Modify the query to reduce the output size. For example during dev work you can use PyRel's top() or bottom() like a LIMIT() function in SQL to get a preview of results. 

with model.query() as select:
    p = Person()
    aggregates.top(10, p)
    response = select(p.id, p.name, p.phone_number, p.email)
response

Alternatively, apply filters to reduce the output size

with model.query() as select:
    p = Person()
    person.age > 30
    response = select(p.id, p.name, p.phone_number, p.email)
response

Export to Snowflake Table

Export the output rather than viewing it in a notebook. This takes a different path that is not subject to the limit. For example use format="snowpark" parameter when querying the model to write the results to a Snowflake table instead. Designate the fully qualified table when pulling the result. The database and schema must already exist 

with model.query(format="snowpark") as select:
    p = Person()
    response = select(p.id, p.name, p.phone_number, p.email)

response.results.write.mode("overwrite").save_as_table("rai_demo.public.person_output_table")

This data can be brought back into RelationalAI for further analysis, by creating a data stream from the new table.

Explanation

This limit of 500 million tuples in the output has been imposed as a safety feature in RelationalAI. 500 million integer pairs can result in roughly 8 GB of blob storage for the client to handle. Thus, this limit was imposed to prevent the client from running out of memory.  Often seeing this large of an output is not useful anyway. Instead it is more useful to redirect the output to another table for storage or further analysis. 

Related Documentation

For more information see these documents.

• RelationalAI Basic Functionality: Writing Results to Snowflake
• RelationalAI Python Reference: top()
• RelationalAI Python Reference: bottom()